What can emotion and abstract words tell us about subjective semantic ratings?

Semantic dimensions such as context availability, imageability and valence, form core components of many theoretical accounts of lexical processing. Typically, normative data for such semantic dimensions are drawn from subjective ratings, however, questions have been raised regarding the reliability and validity of these ratings. In this talk, I will discuss this issue with a focus on context availability norms. Using data collected for another study, we show that context availability ratings required significantly higher rates of data exclusions at the level of both participants and items compared to other variables. In addition, high standard deviations at the item level, indicated a substantial degree of disagreement between participants. Recommendations will be discussed  for norm collection procedures more broadly in order that the validity of such norms can be improved. In particular, clear guidelines are required for data cleaning in order that the reliability of such norms is maximised and to facilitate replication across studies.