{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,30]],"date-time":"2026-03-30T14:30:53Z","timestamp":1774881053779,"version":"3.50.1"},"reference-count":104,"publisher":"Wiley","issue":"2","license":[{"start":{"date-parts":[[2021,12,13]],"date-time":"2021-12-13T00:00:00Z","timestamp":1639353600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/linproxy.fan.workers.dev:443\/http\/onlinelibrary.wiley.com\/termsAndConditions#vor"}],"content-domain":{"domain":["onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["Computer Assisted Learning"],"published-print":{"date-parts":[[2022,4]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:sec><jats:title>Background<\/jats:title><jats:p>Automated Writing Evaluation (AWE) systems to aid writing learning and instruction in primary and secondary education are growing increasingly popular. However, their effectiveness is hardly known. We conducted a systematic review focusing on the effects of these systems providing writing feedback to students in school settings.<\/jats:p><\/jats:sec><jats:sec><jats:title>Objectives<\/jats:title><jats:p>Our goal was to identify and characterize AWE systems tested in the last 20\u2009years for Grades 1\u201312 and examine their impact on text quality and other writing\u2010related outcomes.<\/jats:p><\/jats:sec><jats:sec><jats:title>Methods<\/jats:title><jats:p>The review followed PRISMA guidelines. We identified eight studies reporting the effects on writing of six AWE systems on 1659 students 11\u201317\u2009years of age.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results and conclusions<\/jats:title><jats:p>Our review supported the usefulness of AWE systems for writing learning and instruction. Except for one, all studies showed a positive effect of automated feedback in at least one writing\u2010related measure. The integration of AWE systems into more extensive instructional programs, the amount of writing practice provided to students, the type of the control groups, and the role of teachers are factors influencing their impact on students' writing outcomes.<\/jats:p><\/jats:sec><jats:sec><jats:title>Relevance<\/jats:title><jats:p>Our review generally supported the value of AWE systems in the teaching\/learning process of writing. A closer look into the conditions in which AWE systems are put to practice suggested that they are particularly effective when embedded into comprehensive instructional programs providing ample writing opportunities. Findings from this review expand knowledge on AWE systems as valuable tools to enhance writing in school settings.<\/jats:p><\/jats:sec>","DOI":"10.1111\/jcal.12635","type":"journal-article","created":{"date-parts":[[2021,12,14]],"date-time":"2021-12-14T04:20:20Z","timestamp":1639455620000},"page":"599-620","update-policy":"https:\/\/linproxy.fan.workers.dev:443\/https\/doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":92,"title":["Effectiveness of automated writing evaluation systems in school settings: A systematic review of studies from 2000 to 2020"],"prefix":"10.1111","volume":"38","author":[{"ORCID":"https:\/\/linproxy.fan.workers.dev:443\/https\/orcid.org\/0000-0003-0298-3840","authenticated-orcid":false,"given":"Andreia","family":"Nunes","sequence":"first","affiliation":[{"name":"Faculty of Psychology and Education Sciences University of Porto Porto Portugal"}]},{"ORCID":"https:\/\/linproxy.fan.workers.dev:443\/https\/orcid.org\/0000-0001-8085-314X","authenticated-orcid":false,"given":"Carolina","family":"Cordeiro","sequence":"additional","affiliation":[{"name":"Faculty of Psychology and Education Sciences University of Porto Porto Portugal"}]},{"ORCID":"https:\/\/linproxy.fan.workers.dev:443\/https\/orcid.org\/0000-0002-9903-7289","authenticated-orcid":false,"given":"Teresa","family":"Limpo","sequence":"additional","affiliation":[{"name":"Faculty of Psychology and Education Sciences University of Porto Porto Portugal"}]},{"ORCID":"https:\/\/linproxy.fan.workers.dev:443\/https\/orcid.org\/0000-0002-1487-3596","authenticated-orcid":false,"given":"S\u00e3o Lu\u00eds","family":"Castro","sequence":"additional","affiliation":[{"name":"Faculty of Psychology and Education Sciences University of Porto Porto Portugal"}]}],"member":"311","published-online":{"date-parts":[[2021,12,13]]},"reference":[{"key":"e_1_2_12_3_1","first-page":"316","volume-title":"Handbook of writing research","author":"Allen L. K.","year":"2016"},{"key":"e_1_2_12_4_1","first-page":"181","volume-title":"Handbook of automated essay evaluation: Current applications and new directions","author":"Attali Y.","year":"2013"},{"key":"e_1_2_12_5_1","volume-title":"The psychology of written composition","author":"Bereiter C.","year":"1987"},{"key":"e_1_2_12_6_1","article-title":"The effectiveness of feedback for L1\u2010english and L2\u2010writing development: A meta\u2010analysis","volume":"2011","author":"Biber D.","year":"2011","journal-title":"ETS Research Report RR\u201011\u201005 Princeton, NJ\u2010 ETS"},{"key":"e_1_2_12_7_1","doi-asserted-by":"publisher","DOI":"10.1037\/a0029692"},{"key":"e_1_2_12_8_1","doi-asserted-by":"publisher","DOI":"10.1201\/9781351264808-18"},{"key":"e_1_2_12_9_1","doi-asserted-by":"publisher","DOI":"10.1201\/9781351264808-5"},{"key":"e_1_2_12_10_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10648-020-09530-4"},{"key":"e_1_2_12_11_1","volume-title":"Experimental and quasi\u2010experimental designs for research","author":"Campbell D. T.","year":"1963"},{"key":"e_1_2_12_12_1","doi-asserted-by":"publisher","DOI":"10.17239\/jowr-2020.12.01.02"},{"key":"e_1_2_12_13_1","doi-asserted-by":"publisher","DOI":"10.4000\/aile.4506"},{"key":"e_1_2_12_14_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.edurev.2012.05.002"},{"key":"e_1_2_12_15_1","unstructured":"Conference on College Composition and Communication. (2014).Writing assessment: A position statement.https:\/\/linproxy.fan.workers.dev:443\/http\/www.ncte.org\/cccc\/resources\/positions\/writingassessment."},{"key":"e_1_2_12_16_1","first-page":"1","volume-title":"Genre\u2010based automated writing evaluation for L2 research writing","author":"Cotos E.","year":"2014"},{"key":"e_1_2_12_17_1","first-page":"1","article-title":"Teachers' perspective on using technology as an instructional tool","volume":"24","author":"Davidson L. Y. J.","year":"2014","journal-title":"Research in Higher Education"},{"key":"e_1_2_12_18_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.asw.2012.10.002"},{"issue":"1","key":"e_1_2_12_19_1","first-page":"1","article-title":"An overview of automated scoring of essays","volume":"5","author":"Dikli S.","year":"2006","journal-title":"Journal of Technology, Learning, and Assessment"},{"key":"e_1_2_12_20_1","doi-asserted-by":"publisher","DOI":"10.11139\/cj.28.1.99-134"},{"key":"e_1_2_12_21_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10648-017-9423-5"},{"key":"e_1_2_12_22_1","first-page":"16","volume-title":"Handbook of automated essay evaluation: Current applications and new directions","author":"Elliot N.","year":"2013"},{"key":"e_1_2_12_23_1","first-page":"68","volume-title":"Handbook of automated essay evaluation: Current applications and new directions","author":"Foltz P. W.","year":"2013"},{"key":"e_1_2_12_24_1","doi-asserted-by":"publisher","DOI":"10.2190\/DH8F-QJWM-J457-FQVB"},{"issue":"1","key":"e_1_2_12_25_1","first-page":"1","article-title":"The effect of computers on student writing: A meta\u2010analysis of studies from 1992 to 2002","volume":"2","author":"Goldberg A.","year":"2003","journal-title":"The Journal of Technology, Learning and Assessment"},{"key":"e_1_2_12_26_1","first-page":"187","volume-title":"Handbook of writing research","author":"Graham S.","year":"2006"},{"key":"e_1_2_12_27_1","doi-asserted-by":"publisher","DOI":"10.1080\/00461520.2018.1481406"},{"key":"e_1_2_12_28_1","doi-asserted-by":"publisher","DOI":"10.3102\/0091732X18821125"},{"key":"e_1_2_12_29_1","doi-asserted-by":"publisher","DOI":"10.1002\/trtr.1432"},{"key":"e_1_2_12_30_1","doi-asserted-by":"publisher","DOI":"10.1086\/681947"},{"key":"e_1_2_12_31_1","doi-asserted-by":"publisher","DOI":"10.1037\/0022-0663.99.3.445"},{"key":"e_1_2_12_32_1","doi-asserted-by":"publisher","DOI":"10.1111\/bjet.12864"},{"issue":"6","key":"e_1_2_12_33_1","first-page":"1","article-title":"Utility in a fallible tool: A multi\u2010site case study of automated writing evaluation","volume":"8","author":"Grimes D.","year":"2010","journal-title":"Journal of Technology, Learning, and Assessment"},{"key":"e_1_2_12_34_1","doi-asserted-by":"publisher","DOI":"10.1177\/2372732215624216"},{"key":"e_1_2_12_35_1","doi-asserted-by":"publisher","DOI":"10.3102\/003465430298487"},{"key":"e_1_2_12_36_1","doi-asserted-by":"publisher","DOI":"10.1177\/0741088312451260"},{"key":"e_1_2_12_37_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-29736-7_60"},{"key":"e_1_2_12_38_1","unstructured":"Higgins J. P. T. Thomas J. Chandler J. Cumpston M. Li T. Page M. J. &Welch V. A.(2021).Cochrane Handbook for Systematic Reviews of Interventions version6.2.www.training.cochrane.org\/handbook"},{"key":"e_1_2_12_39_1","doi-asserted-by":"publisher","DOI":"10.1093\/elt\/ccy044"},{"key":"e_1_2_12_40_1","volume-title":"Intelligent tutoring systems: Structure, applications and challenges","author":"Jacovina M. E.","year":"2016"},{"key":"e_1_2_12_41_1","doi-asserted-by":"publisher","DOI":"10.17239\/jowr-2008.01.01.1"},{"key":"e_1_2_12_42_1","doi-asserted-by":"publisher","DOI":"10.1080\/00461520903213600"},{"key":"e_1_2_12_43_1","first-page":"87","volume-title":"Automated essay scoring and annotation of essays with the intelligent essay assessor","author":"Landauer T. K.","year":"2003"},{"key":"e_1_2_12_44_1","doi-asserted-by":"publisher","DOI":"10.1076\/1049-4820(200008)8:2;1-B;FT073"},{"key":"e_1_2_12_45_1","doi-asserted-by":"publisher","DOI":"10.15858\/engtea.75.1.202003.67"},{"key":"e_1_2_12_46_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jslw.2014.10.004"},{"key":"e_1_2_12_47_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.system.2016.02.007"},{"key":"e_1_2_12_48_1","doi-asserted-by":"publisher","DOI":"10.1037\/a0031391"},{"key":"e_1_2_12_49_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.cedpsych.2013.07.004"},{"key":"e_1_2_12_50_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1547-5069.2007.00171.x"},{"key":"e_1_2_12_51_1","doi-asserted-by":"publisher","DOI":"10.1002\/rev3.3114"},{"key":"e_1_2_12_52_1","doi-asserted-by":"publisher","DOI":"10.1002\/jrsm.1411"},{"key":"e_1_2_12_53_1","doi-asserted-by":"publisher","DOI":"10.1598\/RRQ.45.1.4"},{"key":"e_1_2_12_54_1","doi-asserted-by":"publisher","DOI":"10.1016\/S1532-3382(01)70024-3"},{"key":"e_1_2_12_55_1","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pmed.1000097"},{"issue":"2","key":"e_1_2_12_56_1","first-page":"213","article-title":"EssayCritic: Writing to learn with a knowledge\u2010based design critiquing system","volume":"20","author":"M\u00f8rch A.","year":"2017","journal-title":"Educational Technology & Society"},{"key":"e_1_2_12_57_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11145-010-9292-5"},{"key":"e_1_2_12_58_1","first-page":"1","article-title":"Automated writing evaluation (AWE) feedback: A systematic investigation of college students' acceptance","author":"Na Z.","year":"2021","journal-title":"Computer Assisted Language Learning"},{"issue":"1","key":"e_1_2_12_59_1","doi-asserted-by":"crossref","first-page":"113","DOI":"10.1558\/wap.v1i1.113","article-title":"Teacher feedback on writing: Considering the options","volume":"1","author":"Nurmukhamedov U.","year":"2009","journal-title":"Writing & Pedagogy"},{"key":"e_1_2_12_60_1","doi-asserted-by":"publisher","DOI":"10.1186\/s13643-016-0384-4"},{"key":"e_1_2_12_61_1","first-page":"43","volume-title":"Automated essay scoring: A cross\u2010disciplinary perspective","author":"Page E. B.","year":"2003"},{"key":"e_1_2_12_62_1","doi-asserted-by":"publisher","DOI":"10.1080\/10573560308222"},{"key":"e_1_2_12_63_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.cedpsych.2018.07.002"},{"key":"e_1_2_12_64_1","doi-asserted-by":"publisher","DOI":"10.17239\/jowr-2020.12.01.04"},{"key":"e_1_2_12_65_1","doi-asserted-by":"publisher","DOI":"10.29333\/iji.2019.12214a"},{"key":"e_1_2_12_66_1","volume-title":"Department of Special","author":"Peterson\u2010Karlan G. R.","year":"2007"},{"key":"e_1_2_12_67_1","doi-asserted-by":"publisher","DOI":"10.3200\/JOER.101.5.275-286"},{"key":"e_1_2_12_68_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0747-5632(01)00052-8"},{"key":"e_1_2_12_69_1","doi-asserted-by":"publisher","DOI":"10.1037\/a0032340"},{"key":"e_1_2_12_70_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.chb.2016.12.076"},{"key":"e_1_2_12_71_1","volume-title":"Assessing students: How shall we know them?","author":"Rowntree D.","year":"1987"},{"key":"e_1_2_12_72_1","doi-asserted-by":"publisher","DOI":"10.1007\/BF00117714"},{"key":"e_1_2_12_73_1","unstructured":"Sandolo L.. (2010).How can the use of technology enhance writing in the classroom? [Master dissertation St. John Fisher College]. Fisher Digital Publications.https:\/\/linproxy.fan.workers.dev:443\/https\/fisherpub.sjfc.edu\/education_ETD_masters\/194\/."},{"key":"e_1_2_12_74_1","doi-asserted-by":"publisher","DOI":"10.1186\/1472-6947-7-16"},{"key":"e_1_2_12_75_1","doi-asserted-by":"publisher","DOI":"10.1201\/9781351264808-12"},{"key":"e_1_2_12_76_1","doi-asserted-by":"publisher","DOI":"10.1201\/9781351264808-7"},{"key":"e_1_2_12_77_1","doi-asserted-by":"publisher","DOI":"10.4324\/9781410606860"},{"key":"e_1_2_12_78_1","doi-asserted-by":"publisher","DOI":"10.4324\/9780203122761-9"},{"key":"e_1_2_12_79_1","doi-asserted-by":"publisher","DOI":"10.4324\/9780203122761-27"},{"key":"e_1_2_12_80_1","doi-asserted-by":"publisher","DOI":"10.1093\/ajhp\/58.22.2173"},{"key":"e_1_2_12_81_1","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.i4919"},{"key":"e_1_2_12_82_1","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.l4898"},{"key":"e_1_2_12_83_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.asw.2013.11.007"},{"key":"e_1_2_12_84_1","doi-asserted-by":"publisher","DOI":"10.1017\/9781108635547.009"},{"key":"e_1_2_12_85_1","doi-asserted-by":"publisher","DOI":"10.29140\/jaltcall.v13n2.215"},{"key":"e_1_2_12_86_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.compedu.2011.06.008"},{"key":"e_1_2_12_87_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.amj.2006.09.001"},{"key":"e_1_2_12_88_1","first-page":"324","volume-title":"Handbook of writing research","author":"Troia G. A.","year":"2006"},{"key":"e_1_2_12_89_1","doi-asserted-by":"publisher","DOI":"10.1558\/calico.v32i1.25961"},{"key":"e_1_2_12_90_1","volume-title":"UNESCO and education: Everyone has the right to education","author":"United Nations Educational, Scientific and Cultural Organization, UNESCO","year":"2011"},{"key":"e_1_2_12_91_1","doi-asserted-by":"publisher","DOI":"10.1287\/isre.11.4.342.11872"},{"key":"e_1_2_12_92_1","doi-asserted-by":"publisher","DOI":"10.2307\/41410412"},{"key":"e_1_2_12_93_1","volume-title":"Mind in society: The development of higher psychological processes","author":"Vygotsky L. S.","year":"1978"},{"key":"e_1_2_12_94_1","doi-asserted-by":"publisher","DOI":"10.1207\/s1532690xci2203_3"},{"key":"e_1_2_12_95_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.asw.2020.100449"},{"issue":"1","key":"e_1_2_12_96_1","first-page":"79","article-title":"Effects of an automated writing evaluation program: Student experiences and perceptions","volume":"12","author":"Wang P.\u2010L.","year":"2015","journal-title":"Electronic Journal of Foreign Language Teaching"},{"key":"e_1_2_12_97_1","doi-asserted-by":"publisher","DOI":"10.1080\/09588221.2012.655300"},{"key":"e_1_2_12_98_1","doi-asserted-by":"publisher","DOI":"10.1558\/wap.v6i2.223"},{"key":"e_1_2_12_99_1","doi-asserted-by":"publisher","DOI":"10.1080\/15544800701771580"},{"key":"e_1_2_12_100_1","doi-asserted-by":"publisher","DOI":"10.1191\/1362168806lr190oa"},{"key":"e_1_2_12_101_1","first-page":"36","volume-title":"Handbook of automated essay evaluation: Current applications and new directions","author":"Weigle S. C.","year":"2013"},{"key":"e_1_2_12_102_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11145-018-9836-7"},{"key":"e_1_2_12_103_1","doi-asserted-by":"publisher","DOI":"10.1037\/edu0000168"},{"key":"e_1_2_12_104_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11145-016-9695-z"},{"key":"e_1_2_12_105_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.compedu.2016.05.004"},{"key":"e_1_2_12_106_1","doi-asserted-by":"publisher","DOI":"10.1177\/0735633119830764"}],"container-title":["Journal of Computer Assisted Learning"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/linproxy.fan.workers.dev:443\/https\/onlinelibrary.wiley.com\/doi\/pdf\/10.1111\/jcal.12635","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/linproxy.fan.workers.dev:443\/https\/onlinelibrary.wiley.com\/doi\/full-xml\/10.1111\/jcal.12635","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/linproxy.fan.workers.dev:443\/https\/onlinelibrary.wiley.com\/doi\/pdf\/10.1111\/jcal.12635","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,11,14]],"date-time":"2023-11-14T02:02:12Z","timestamp":1699927332000},"score":1,"resource":{"primary":{"URL":"https:\/\/linproxy.fan.workers.dev:443\/https\/onlinelibrary.wiley.com\/doi\/10.1111\/jcal.12635"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,12,13]]},"references-count":104,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2022,4]]}},"alternative-id":["10.1111\/jcal.12635"],"URL":"https:\/\/linproxy.fan.workers.dev:443\/https\/doi.org\/10.1111\/jcal.12635","archive":["Portico"],"relation":{},"ISSN":["0266-4909","1365-2729"],"issn-type":[{"value":"0266-4909","type":"print"},{"value":"1365-2729","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,12,13]]},"assertion":[{"value":"2021-04-30","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-11-21","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-12-13","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}