What questions to use for the Dyslexia Questionnaire

Over the years we have had a number of different types of questionnaires for use in our Dyslexia Questionnaire, which has either been as a standalone, or with other SpLDs. The question is, which is “best”. Of course, there can be no best really, since it comes down to a question of validity, and how can you be sure of the validity of a dyslexia tool when you cannot decide the criteria to know if somebody is dyslexic or not.

I would suggest the main approaches have been:

  1. Smythe and Everatt 2000 – This has been widely used, including as the default for the British Dyslexia Association. It was originally a survey of the responses to over 50 questions by more than 300 individuals. We used a regression technique to provide weightings to each of the most important questions based on a “self-diagnosis” (i.e. have you been diagnosed as dyslexic?) However, the results were not published in a peer-review paper as we were not able to confirm that those diagnoses were for real.
  2. Symptom-based checklist – If the major defining characteristics are “a difficulty in the acquisition of reading, writing and spelling”, then all you have to identify (and preferably prove) is that difficulty in the acquisition. (Note that this is not a difficulty, but a difficulty in the acquisition of those skills.)
Posted in Uncategorised | Comments Off on What questions to use for the Dyslexia Questionnaire

Content of the new New Word Spelling Test

It would be inappropriate to put all the items of the test up here. But I can share the main “characteristics” of the test.

  1. There are 15 items in all.
  2. Seven are CVC words, four are CVCC, and two are CCVC. The others are CVCV and CCVCCC.
  3. There are four words with the vowel “e”, three for “a”, “i” and “o”, and two use “u”. This is in order of most common vowels in the English language.
  4. Thirteen different first letters are used.
  5. Four consonant blends are found at the beginning of the words.
  6. Five consonant blends are found at the end of words.
  7. There are ten different last letters.

As far as possible, the words have a sound where there is one “most probable” spelling. However, the results will show what the user thinks, and may lead to alternatives being made viable options.

All derived words have been checked on the internet, highlighting the difficulty of finding “unused” letter combinations. Some have been found to be acronyms, others are non-English words and some are company names. But most will be unknown to the typical reader.

All these new words conform to what I would describe as “not looking out of context in a modern text”.

Posted in Uncategorised | Comments Off on Content of the new New Word Spelling Test

Making the new New Word Spelling test

Let’s start with a few basic questions:

What is a “New Word Spelling Test”? Basically, it is a spelling test whereby the individual is asked to spell which they are not familiar with, either because they are very rare or made-up. Thus they cannot have practiced the spelling. Hence it can be very useful in the evaluation of an individuals’ skill in the area of phonics.

So why is this important? Imagine starting a technical course where you need to learn a lot of new terminology. How do you know how to spell a word you hear in class if you do not have the basic skills learned through phonics? The answer is that you will have great difficulty, and guess is often not an option. As a consequence, you may have great natural skill in the subject, but your writing skills let you down such that your actual ability is not reflected in your exam scores.

Why does Profiler need a new New Word Spelling Test? Profiler has had several version of the New Word Spelling Test (NWST) since the start. These have evolved to suit diverse (and emerging) needs. However, there are a number of issues with such tests, the chief ones of which are:

    1. They do not measure a single construct – If you look at the psychometric properties of a NWST, you will find that the reliability (alpha) is not great (0.80 for UK data and 0.71 for SA data), even if you have large numbers of data points. The reason is that these tests, unless they are very long, do not measure a single construct. That is, they do not measure “phonics” per se, but a series of factors, exacerbated by the nature of the English written language. Think of it as needing a separate test for not only all the vowels, but also each of the vowels sounds. And that is before we think about the Rules of English Spelling (not to mention the exceptions), and the diversity of ways to spell a single (new) word. To have a single test that covers all that would be unworkably long.
    2. Multilingualism in a community leads to diverse spellings – Experience in South Africa shows that the first language of an individual can significantly affect the way they spell a word. One good example is the (new word) mip (to rhyme with dip, pip, sip, tip etc). Results showed that more SA students spell this word (spoken by a UK English person) wrote map than mip. However, consider that the pronunciation is the same (or very close to) the word used to represent a diagram that shows you how to drive from A to B – a map! So can we say that the spelling map is wrong? No! But that makes the norming process, at best, problematic. Do we make a norm for every language? No, because then there are even more complications, such as what do you expect the norm to be for a Grade 5 child whose first language is Zulu, who has been in an Afrikaans school since Grade 1 and is being tested in English?

So does this mean that because of these complications, there is no place for a NWST? It means that we have to understand the context and the use of the test, and offer information accordingly. Put another way, as a criterion-based test, it is a very useful tool.

Criteria for the new NWST

In order to ensure we have a new test that is fit-for-purpose for its wider use, I set up a series of criteria that it had to conform to. Below are these criteria, and the justification for each.

The test must be short – Teachers (and students) want quick answers. This new test will initially be open-ended but data analysis will suggest the best cut-off time for this test. Results below are for SA students, with the time taken to complete the first eight questions, which are the monosyllabic ones. The total number of questions is twice this. The initial cut-off will be 20 minutes, or 1200 seconds.

The graph below shows that SA students take almost twice as long on average to complete these first eight questions (out of 16). Ages are comparable.

It should focus on simple sound-letter correspondence – There are many different levels and combinations one could work with, some of which have been used in previous modules. But experience suggest that the results over and above the monosyllable has been little used. Therefore the intent here is to get back to basics, and check at the single letter level and to a lesser extent, a few combinations of consonants – though how do you choose which ones to use?

All words should be monosyllabic – That means there should be minimal reliance on working memory.

It should cater for diverse spellings – There can be more than one correct spelling. For example, the three spelling of fote, foat and phote all sound the same and are equally valid.

There should be a good representation of the vowels – From previous research, the vowel sound is often the most problematic, especially in multilingual cultures. Therefore there should be adequate coverage, with more emphasis on the more common vowels.

Error analysis should be offered for the most common errors – There are a limited number of errors that can be made, mostly related to vowels. However, there can be confusion between some consonants, such as p / b.

The test is criterion, not normed, based – The idea for this test is to know if the person can get the right answer, and not necessarily how well they can do the test compared to others. Therefore, norms are not required. This does not mean they will not be available in future. But not in the short term.

The order of items will be in order of difficulty – This is required so that it is possible to use an exit system, such as 6 errors and out. This cuts the time. Implementation will only be possible once a minimum data set is available.

The test should also be a form of hearing test – As far as possible, the consonants should use a wide range of sounds. Of course, without repetitions, it is not possible to be sure. But it allows the teacher / tutor to review and make informed decisions.

Future research may look at patterns of errors – At this level, the analysis is at a one to one level. That is, there is no current ability to look at, say, the errors in one vowel type, of final consonant difficulties. This will be possible in future.

Posted in Uncategorised | Comments Off on Making the new New Word Spelling test

Reading Summary

These questions were designed to be the Reading related questions, combining questions that were previously used, ones that appeared to have face validity and ones that fellow practitioners also noted. The question I want to address here are:

  • how do we compare and contract the questions
  • should some of the questions be deleted
  • what are the most predictive questions

The simple answer is that no matter which analysis you attempt, there will be a mixture of uncertainty, qualitative interpretation and questionable statistical validity. Or to put it another way, whatever I do, somebody will tell me I am wrong. So starting from that point, where do we go?

Firstly, I decided to review the questions and give the seven reading-related questions a ranking of up to five stars which is nothing more than my subjective statement of what looked good and what I would include in future questionnaires. And since it is a personal subjective view, it cannot be wrong, only (maybe) misguided. Above is my brief personal reflection, which ends up with two questions, namely

  • How often do you confuse visually similar words, such as cat and cot?
  • How difficult do you find reading aloud?

One of the problems within any analysis of this kind is that a Likert scale is being used. This means most statistical analyses are not valid, since, for example, we cannot know if the difference between each of these answers progresses in a linear way, or even if everybody was consistent in the “linearisation” of the answers. So comparing means or even looking at correlations is arguably suspect.

However, there are two analyses which I do intend to do in future.

  1. Do a table that plots the two 4-star questions against each other, such that it is not a correlation but contingency table, which may help understand the relationship (can we call it a pseudo-correlation?) between the two questions.
  2. Using the self-reporting, look at the level of Type 1 and Type 2 errors that would occur is these two questions were used. Do I mean only use two questions instead of the seven reading questions for that section, or do I mean use only two questions to identify dyslexic individuals? You will need to read the next entry to answer that one.
Posted in Uncategorised | Comments Off on Reading Summary

How fast do you think you read compared to most people?

This question was not one used in the earlier (2000) questionnaire but was included to widen the range of potential questions, and to see if what has been used by some researchers is a valid indicator or dyslexia, or should at least be part of the screening process.

The overlaps on this mean it has very limited predictive capability (I will give it 2-stars), but may help in a discussion with support.

Posted in Uncategorised | Comments Off on How fast do you think you read compared to most people?

How difficult is it for you to understand what you have read, e.g. when reading a book or paper, do you have to re-read sections to understand the content?

Reading comprehension is a common item in dyslexia screening questionnaires, not surprisingly. However, once again the overlap is significant, and therefore the predictive quality is limited. Or perhaps a more realistic (correct?) way to put it is that non-dyslexic individuals also have problems of this type.

If we look at the graph above, we can see that of all those who reported having trouble “most of the time”, something like 15% said they were not dyslexic.

Conclusion

Not as good as some as a predictor. Probably its best use is in the discussion of support, which is highly context dependent. This is because the first question must be “What sort of reading are you referring to?”

Posted in Uncategorised | Comments Off on How difficult is it for you to understand what you have read, e.g. when reading a book or paper, do you have to re-read sections to understand the content?

How often do you avoid reading books because of the effort or difficulty in doing so?

This seems like a logical question which practitioners find in most dyslexic adults. After all, if you are dyslexic, it would seem logical

The trouble with this question is that everybody can interpret it in their own way. So some may see it as a challenge for reading the words (e.g. dyslexic readers), whilst others see the problem as comprehension (e.g. on a university course).  Hence it is not surprising there is such an overlap, and potential to make erroneous conclusions if you make the

Conclusion

This question is not a prime candidate for a screener going forward.

Posted in Uncategorised | Comments Off on How often do you avoid reading books because of the effort or difficulty in doing so?

How often do you lose your place or miss out lines when reading?

We can all miss out lines when reading, and that creates problems with understanding, to say the least. But it seems logical that dyslexic individuals would do it more often.

But the honest answer is that in the earlier work in 2000, this was not one of the questions which came up as highly predictive. That said, there were only five questions that were highly predictive. But who would believe a checklist of just five questions?

Conclusion

Whilst this may not be one of the most predictive questions, it can help both in terms of supporting an individual, and for self-awareness.

Posted in Uncategorised | Comments Off on How often do you lose your place or miss out lines when reading?

How difficult do you find reading aloud?

Reading aloud in front of others can be a major difficulty for most dyslexic individuals. Reading to oneself can be problematic, but to show to others ones can

Once again, there appears to be a steady progression, in line with expectations from working with a lot of practitioners as well as talking to many dyslexic individuals.

Posted in Uncategorised | Comments Off on How difficult do you find reading aloud?

DQ – How often do you confuse visually similar words, such as cat and cot?

This question reflects the problem that many reluctant readers suffer from, which is that sometimes they resort to a quick visual scan and make rapid decisions based on what is seen. The question is why? It may be they are trying to read quickly due to pressure of some kind (e.g. peers or time limitations). And the decision can be because they think they recognise it and run with that decision.

So let’s look at the numbers.

This is very much a progression, which make is an ideal candidate for the screening tool.

Conclusion

  1. This question focussed on the simple word (consonant-vowel-consonant), and as a self-assessment questionnaire, one can but assume the person answering the questions also relates this to longer questions such as reading quickly as quietly or refraction as reflection. It would be good to have a test whereby the person has (say) just a second to choose between two words to evaluate this as a cognitive skill. Whilst it may sound easy, in a speeded test there are other issues such as coordination and the decision-making process.
  2. Evidence suggests that this question is (relatively) highly predictive of  dyslexia, and it is not surprising it has higher weighting in screeners.
Posted in Uncategorised | Comments Off on DQ – How often do you confuse visually similar words, such as cat and cot?