The article is based on students’ compositions from the Lower Secondary School Students Competence Assessment 2011 and discusses the sources of problems in assessing the linguistic accuracy of students’ written work. The reasons are both strictly linguistic in nature (e.g. it not always being clear how to classify certain textual phenomena) and extralinguistic (e.g. different examiners inconsistently evaluate the same errors). Examiners’ decisions are important in the lower secondary school examination as a quantitative scale is used to determine students’ final scores according to the number of errors identified by the examiner. The analysis presented demonstrates the unreliability of this approach. To mollify this situation, we should conduct teacher support activities and modify the method of evaluating linguistic aspects of students’ work in the public examination system.