Abstract
Objectives
Assess the extent to which measurement error in police recorded crime rates impact the estimates of regression models exploring the causes and consequences of crime.
Methods
We focus on linear models where crime rates are included either as the response or as an explanatory variable, in their original scale or log-transformed. Two measurement error mechanisms are considered, systematic errors in the form of under-recorded crime, and random errors in the form of recording inconsistencies across areas. The extent to which such measurement error mechanisms impact model parameters is demonstrated algebraically using formal notation, and graphically using simulations.
Results
The impact of measurement error is highly variable across different settings. Depending on the crime type, the spatial resolution, but also where and how police recorded crime rates are introduced in the model, the measurement error induced biases could range from negligible to severe, affecting even estimates from explanatory variables free of measurement error. We also demonstrate how in models where crime rates are introduced as the response variable, the impact of measurement error could be eliminated using log-transformations.
Conclusions
The validity of a large share of the evidence base exploring the effects and consequences of crime is put into question. In interpreting findings from the literature relying on regression models and police recorded crime rates, we urge researchers to consider the biasing effects shown here. Future studies should also anticipate the impact in their findings and employ sensitivity analysis if the expected measurement error induced bias is non-negligible.
Similar content being viewed by others
Notes
The CSEW sampling approach is designed to enable the calculation of reliable victimisation estimates at the PFA level, with an average sample of 1,096 respondents in each area (min = 917, max = 4023). PFA is an UK spatial unit commonly used in the literature (Abramovaite et al. 2019; Han et al. 2013; Machin and Meghir 2004), encompassing 1.3 million people on average, which makes them similar to states and large counties in the US (Barnett 1981; Philipson and Posner 1996).
Home Office data is available here: https://www.gov.uk/government/statistics/police-recorded-crime-open-data-tables.
The City of London is primarily a business and financial centre with a small resident population of approximately 10,000 but a large day‐time population leading to artificially high crime rates.
UCR data is available here: https://ucr.fbi.gov/crime-in-the-u.s/2019/crime-in-the-u.s.-2019/topic-pages/tables/table-20, NCHS data is available here: https://wonder.cdc.gov/controller/saved/D76/D99F056.
Proof of the variance being unaffected by a change of origin:
$$S_{{X^{*} }}^{2} = \frac{{\sum \left( {X^{*} - \overline{X}^{*} } \right)^{2} }}{n - 1} = \frac{{\sum \left( {X + u - \left( {\overline{X} + u} \right)} \right)^{2} }}{n - 1} = \frac{{\sum \left( {X - \overline{X}} \right)^{2} }}{n - 1} = S_{X}^{2} .$$Proof of the covariance being unaffected by a change of origin:
$$S_{{X^{*} ,Y}} = \frac{{\Sigma (X^{*} - \overline{X}^{*} ) (Y^{*} - \overline{Y}^{*} )}}{n - 1} = \frac{{\Sigma \left( {X + u - \left( {\overline{X} + u} \right)} \right) (Y^{*} - \overline{Y}^{*} )}}{n - 1} = \frac{{\Sigma \left( {X - \overline{X}} \right) (Y^{*} - \overline{Y}^{*} )}}{n - 1} = S_{X,Y} .$$Proof of the variance being affected by a change in scale:
$$S_{{X^{*} }}^{2} = \frac{{\sum \left( {X^{*} - \overline{X}^{*} } \right)^{2} }}{n - 1} = \frac{{\sum \left( {Xu - \overline{X}u} \right)^{2} }}{n - 1} = \frac{{u^{2} \sum \left( {X - \overline{X}} \right)^{2} }}{n - 1} = u^{2} S_{X}^{2}$$Proof of the covariance being affected by a change in scale:
$$S_{{X^{*} ,Y}} = \frac{{\Sigma (X^{*} - \overline{X}^{*} ) (Y^{*} - \overline{Y}^{*} )}}{n - 1} = \frac{{\Sigma \left( {Xu - \left( {\overline{X}u} \right)} \right) (Y^{*} - \overline{Y}^{*} )}}{n - 1} = \frac{{u\Sigma \left( {X - \overline{X}} \right) (Y^{*} - \overline{Y}^{*} )}}{n - 1} = uS_{X,Y} .$$The R code used can be found here, https://osf.io/kv3sc/.
References
Abramovaite J, Bandyopadhyay S, Bhattacharya S, Cowen N (2019) Alternatives to custody: evidence from police force areas in England and Wales. Br J Criminol 59(4):800–822
Ashby MP (2020) Changes in police calls for service during the early months of the 2020 coronavirus pandemic. Polic J Policy Pract 14(4):1054–1072
Barnett A (1981) The deterrent effect of capital punishment: a test of some recent studies. Oper Res 29(2):346–370
Baumer EP (2002) Neighborhood disadvantage and police notification by victims of violence. Criminology 40(3):579–616
Berg MT, Lauritsen JL (2016) Telling a similar story twice? NCVS/UCR convergence in serious violent crime rates in rural, suburban, and urban places (1973–2010). J Quant Criminol 32:61–87
Biderman AD, Reiss AJ Jr (1967) On exploring the ‘dark figure’ of crime. Ann Am Acad Pol Soc Sci 374(1):1–15
Biewen E, Nolte S, Rosemann M (2008) Perturbation by multiplicative noise and the simulation extrapolation method. Adv Stat Anal 92(4):375–389
Birks D, Townsley M, Stewart A (2012) Generative explanations of crime: using simulation to test criminological theory. Criminology 50(1):221–254
Blackwell M, Honaker J, King G (2017) A unified approach to measurement error and missing data: overview and applications. Sociol Methods Res 46(3):303–341
Boivin R, Cordeau G (2011) Measuring the impact of police discretion on official crime statistics: a research note. Police Q 14(2):186–203
Bottoms AE, Mawby RI, Walker MA (1987) A localised crime survey in contrasting areas of a city. Br J Criminol 23(2):125–154
Bowers KJ, Johnson SD (2005) Domestic burglary repeats and space-time clusters: the dimensions of risk. Eur J Criminol 2(1):67–92
Brantingham P (2018) The logic of data bias and its impact on place-based predictive policing. Ohio State J Crim Law 15(2):473–486
Browning CR, Cagney KA, Iveniuk J (2012) Neighborhood stressors and cardiovascular health: crime and C-reactive protein in Dallas, USA. Soc Sci Med 75(7):1271–1279
Buil-Gil D, Moretti A, Langton SH (2021) The accuracy of crime statistics: assessing the impact of police data bias on geographic crime analysis. J Exp Criminol 6:66
Buil-Gil D, Medina J, Shlomo N (2021b) Measuring the dark figure of crime in geographic areas: small area estimation from the Crime Survey for England and Wales. Br J Criminol 61(2):364–388
Burrows J, Tarling R, Mackie A (2000) Review of police force’s crime recording practices. Home Office Research Study
Cantor D, Cohen LE (1980) Comparing measures of homicide trends: methodological and substantive differences in the vital statistics and Uniform Crime Report time series (1933–1975). Soc Sci Res 9:121–145
Cantor D, Land KC (1985) Unemployment and crime rates in the post-World War II United States: a theoretical and empirical analysis. Am Sociol Rev 66:317–332
Caplan JM, Kennedy LW, Miller J (2011) Risk terrain modeling: brokering criminological theory and GIS methods for crime forecasting. Just Q 28(2):360–381
Carroll RJ, Ruppert D, Stefanski LA, Crainiceanu CM (2006) Measurement error in nonlinear models: a modern perspective. CRC Press, Boca Raton
Cernat A, Buil-Gil D, Brunton-Smith I, Pina-Sánchez J, Murrià-Sangenís M (2021) Estimating crime in place: moving beyond residence location. Crime Delinq 6:66
Cho JT, Park J (2017) Exploring the effects of CCTV upon fear of crime: a multi-level approach in Seoul. Int J Law Crime Just 49:35–45
Cohen LE, Felson M (1979) Social change and crime rate trends: a routine activity approach. Am Sociol Rev 66:588–608
Coleman C, Moynihan J (1996) Understanding crime data: haunted by the Dark Figure. Open University Press, Buckingham
Duncan TE, Duncan SC, Okut H, Strycker LA, Hix-Small H (2003) A multilevel contextual model of neighborhood collective efficacy. Am J Community Psychol 32(3–4):245–252
Ellis L, Farrington DP, Hoskin AW (2019) Handbook of crime correlates. Academic Press, London
Eterno JA, Verma A, Silverman EB (2016) Police manipulations of crime reporting: Insiders’ revelations. Just Q 33(5):811–835
Fajnzylber P, Lederman D, Loayza N (2002) Inequality and violent crime. J Law Econ 45(1):1–39
Farrell G, Pease K (2003) Measuring and interpreting repeat victimization using police data: an analysis of burglary data and policy for Charlotte, North Carolina. Crime Preven Stud 16:265–289
Fuller WA (2009) Measurement error models. Wiley, New York
Gibson J, Kim B (2008) The effect of reporting errors on the cross-country relationship between inequality and crime. J Dev Econ 87(2):247–254
Glewwe P (2007) Measurement error bias in estimates of income and income growth among the poor: analytical results and a correction formula. Econ Dev Cult Change 56:163–189
Goulas E, Zervoyianni A (2013) Economic growth and crime: Does uncertainty matter? Appl Econ Lett 20(5):420–427
Greenberg DF (2001) Time series analysis of crime rates. J Quant Criminol 17(4):291–327
Groff ER, Johnson SD, Thornton A (2019) State of the art in agent-based modeling of urban crime: an overview. J Quant Criminol 35(1):155–193
Gustafson P (2003) Measurement error and misclassification in statistics and epidemiology: impacts and Bayesian adjustments. CRC Press, Boca Raton
Han L, Bandyopadhyay S, Bhattacharya S (2013) Determinants of violent and property crimes in England and Wales: a panel data analysis. Appl Econ 45(34):4820–4830
Harmon MG (2013) “Fixed” sentencing: the effect on imprisonment rates over time. J Quant Criminol 29(3):369–397
Hart TC, Rennison C (2003) Reporting crime to the police, 1992–2000. Special Report. Bureau of Justice Statistics. https://static.prisonpolicy.org/scans/bjs/rcp00.pdf
Her Majesty Inspectorate of Constabulary (2014) Crime-recording: making the victim count—The final report of an inspection of crime data integrity in police forces in England and Wales. Justice Inspectorates. https://www.justiceinspectorates.gov.uk/hmicfrs/wp-content/uploads/crime-recording-making-the-victim-count.pdf
Hibdon J, Telep CW, Groff ER (2017) The concentration and stability of drug activity in Seattle, Washington using police and emergency medical services data. J Quant Criminol 33(3):497–517
Hughes LA, Schaible LM, Kephart T (2021) Gang graffiti, group process, and gang violence. J Quant Criminol 6:66
Jackson J, Bradford B, Stanko B, Hohl K (2013) Just authority? Trust in the police in England and Wales, Routledge, Abingdon
Keels M, Duncan GJ, DeLuca S, Mendenhall R, Rosenbaum J (2005) Fifteen years later: can residential mobility programs provide a long-term escape from neighborhood segregation, crime, and poverty. Demography 42(1):51–73
Klinger DA, Bridges GS (1997) Measurement error in calls-for-services as an indicator of crime. Criminology 35(4):705–726
Krahn H, Kennedy LW (1985) Producing personal safety: the effects of crime rates, police force size, and fear of crime. Criminology 23(4):697–710
Laufs J, Bowers K, Birks D, Johnson SD (2020) Understanding the concept of ‘demand’ in policing: a scoping review and resulting implications for demand management. Polic Soc 6:66
Lee MR, Ousey GC (2005) Institutional access, residential segregation, and urban black homicide. Sociol Inq 75(1):31–54
Levitt SD (1998) Why do increased arrest rates appear to reduce crime: deterrence, incapacitation, or measurement error? Econ Inq 36(3):353–372
Lohr SH (2019) Measuring crime: behind the statistics. CRC Press, Boca Raton
Machin S, Meghir C (2004) Crime and economic incentives. J Hum Resour 39(4):958–979
Maltz MD, Targonski J (2002) A note on the use of country-level UCR data. J Quant Criminol 18(3):297–318
Martin RA, Legault LR (2005) Systematic measurement error with state-level crime data: Evidence from the “more guns, less crime” debate. J Res Crime Delinq 42(2):187–210
Matsueda RL, Kreager DA, Huizinga D (2006) Deterring delinquents: a rational choice model of theft and violence. Am Sociol Rev 71(1):95–122
McCandless R, Feist A, Allan J, Morgan N (2016) Do initiatives involving substantial increases in stop and search reduce crime? Assessing the impact of Operation BLUNT 2. Home Office Report. https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/508661/stop-search-operation-blunt-2.pdf
McCarthy M, Porter L, Townsley M, Alpert G (2019) Influence of community characteristics on serious police use of force events in an Australian policing jurisdiction: a test of minority threat, social disorganisation, and ecological contamination theories. Polic Soc 29(9):1091–1108
Messer LC, Kaufman JS, Dole N, Savitz DA, Laraia BA (2006) Neighborhood crime, deprivation, and preterm birth. Ann Epidemiol 16(6):455–462
Miethe TD, Hughes M, McDowall D (1991) Social change and crime rates: an evaluation of alternative theoretical approaches. Soc Forces 70(1):165–185
Morenoff JD, Sampson RJ (1997) Violent crime and the spatial dynamics of neighborhood transition: Chicago, 1970–1990. Soc Forces 76(1):31–64
Neumayer E (2005) Inequality and violent crime: evidence from data on robbery and violent theft. J Peace Res 42(1):101–112
Novick MR (1966) The axioms and principal results of classical test theory. J Math Psychol 3(1):1–18
Nugent W, Graycheck L, Basham R (2000) A devil hidden in the details: the effects of measurement error in regression analysis. J Soc Serv Res 27:53–75
Oberski DL, Kirchner A, Eckman S, Kreuter F (2017) Evaluating the quality of survey and administrative data with generalized multitrait-multimethod models. J Am Stat Assoc 112(520):1477–1489
ONS (2015) User guide to crime statistics for England and Wales. Office for National Statistics
ONS (2022) Crime in England and Wales: year ending September 2021. Office for National Statistics
Osborn DR, Tseloni A (1998) The distribution of household property crimes. J Quant Criminol 14(3):307–330
Osgood DW (2000) Poisson-based regression analysis of aggregate crime rates. J Quant Criminol 16:21–43
Pavlopoulos D, Pankowska PK, Bakker BF, Oberski D (2020) Modelling error dependence in categorical longitudinal data. In: Cernat A, Sakshaug J (eds) Measurement error in longitudinal data. Oxford University Press, Oxford
Pepper J, Petrie C, Sullivan S (2010) Measurement error in criminal justice data. In: Piquero AR, Weisburd D (eds) Handbook of quantitative criminology. Springer, Berlin
Philipson TJ, Posner RA (1996) The economic epidemiology of crime. J Law Econ 39(2):405–433
Pickles A, Pickering K, Taylor C (1996) Reconciling recalled dates of developmental milestones, events and transitions: a mixed generalized linear model with random mean and variance functions. J R Stat Soc Ser A 66:225–234
Pina-Sánchez J (2016) Adjustment of recall errors in duration data using SIMEX. Metodološki Zvezki-Adv Methodol Stat 13(1):27–58
Pina-Sánchez J, Koskinen J, Plewis I (2019) Adjusting for measurement error in retrospectively reported work histories: an analysis using Swedish register data. J Off Stat 35(1):203–229
Pudney S, Deadman D, Pyle D (2000) The relationship between crime, punishment and economic conditions: is reliable inference possible when crimes are under-recorded? J R Stat Soc A 163(1):81–97
Regoeczi WC, Banks D, Planty M, Langton L, Warner M (2014) The nation’s two measures of homicide. Program Report, Bureau of Justice Statistics, U.S. Department of Justice
Reisig MD, Parks RB (2000) Experience, quality of life, and neighborhood context: a hierarchical analysis of satisfaction with police. Just Q 17(3):607–630
Sampson RJ, Raudenbush SW, Earls F (1997) Neighborhoods and violent crime: a multilevel study of collective efficacy. Science 277(5328):918–924
Schneider AL (1981) Methodological problems in victim surveys and their implications for research in victimology. J Crim Law Criminol 72(2):818–838
Skinner C, Humphreys K (1999) Weibull regression for lifetimes measured with error. Lifetime Data Anal 5:23–37
Skogan WG (1977) Dimensions of the dark figure of unreported crime. Crime Delinq 23(1):41–50
Sobol JJ, Wu Y, Sun IY (2013) Neighborhood context and police vigor: a multilevel analysis. Crime Delinq 59(3):344–368
Solymosi R, Buil-Gil D, Vozmediano L, Guedes IS (2020) Towards a place-based measure of fear of crime: a systematic review of app-based and crowdsourcing approaches. Environ Behav 6:66
Stefanski LA, Cook JR (1995) Simulation-extrapolation: the measurement error jackknife. J Am Stat Assoc 90(432):1247–1256
Sutherland A, Brunton-Smith I, Jackson J (2013) Collective efficacy, deprivation and violence in London. Br J Criminol 53(6):1050–1074
Sutherland A, Strang L, Stepanek M, Giacomantonio C, Boyle A, Strang H (2021) Tracking violent crime with ambulance data: How much crime goes uncounted? Camb J Evid Based Polic 6:66
Tarling R, Morris K (2010) Reporting crime to the police. Br J Criminol 50(3):474–490
van de Weijer SGA, Leukfeldt R, Bernasco W (2019) Determinants of reporting cybercrime: a comparison between identity theft, consumer fraud, and hacking. Eur J Criminol 16(4):486–508
Vandeviver C, Neutens T, Van Daele S, Geurts D, Vander Beken T (2015) A discrete spatial choice model of burglary target selection at the house-level. Appl Geogr 64:24–34
Vollaard B, Hamed J (2012) Why the police have an effect on violent crime after all: evidence from the British Crime Survey. J Law Econ 55(4):901–924
von Hofer H (2000) Crime statistics as constructs: the case of Swedish rape statistics. Eur J Crim Policy Res 8:77–89
Whitworth A (2012) Inequality and crime across England: a multilevel modelling approach. Soc Policy Soc 1(14):48–68
Witt R, Witte A (2000) Crime, prison, and female labor supply. J Quant Criminol 16(1):69–85
Xie M, Baumer EP (2019) Neighborhood immigrant concentration and violent crime reporting to the police: a multilevel analysis of data from the National Crime Victimization Survey. Criminology 57(2):237–267
Xie M, Lauritsen JL (2012) Racial context and crime reporting: a test of Black’s stratification hypothesis. J Quant Criminol 28(2):265–293
Yang SM, Hinkle JC, Wyckoff LA (2018) Using multitrait-multimethod (MTMM) techniques to examine the convergent and discriminant validity of social disorder. J Res Crime Delinq 55(5):571–608
Yesberg J, Brunton-Smith I, Bradford B (2021) Police visibility, trust in police fairness, and collective efficacy: a multilevel structural equation model. https://osf.io/preprints/socarxiv/k7ren/
Young JT (2014) A sensitivity analysis of egocentric measures of peer delinquency to latent homophily: a research note. J Quant Criminol 30(3):373–387
Zhao JS, Lawton B, Longmire D (2015) An examination of the micro-level crime–fear of crime link. Crime Delinq 61(1):19–44
Funding
This work is supported by the Secondary Data Analysis Initiative of the Economic and Social Research Council (Grant Ref: ES/T015667/1).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Below is the link to the electronic supplementary material.
Appendix: Specific Offences Used to Define Broader Crime Types in Table 1
Appendix: Specific Offences Used to Define Broader Crime Types in Table 1
CSEW 2018–2019 | NCVS 2017–2020* | ||||
---|---|---|---|---|---|
Crime type | Cases reported in the interview | % known to police (weighted) | Crime type | Cases reported in the interview | % known to police (weighted) |
Violent crime | 1979 | 38.8 | Violent crime | 516 | 46.6 |
Hit with fists or weapon | 538 | 46.6 | �Assault | 200 | 49.5 |
�Threaten to use force or violence on you | 1319 | 36.4 | �Attempted assault | 299 | 44.7 |
�Sexually assaulted | 95 | 28.2 | �Rape | 8 | ** |
�Violent from household member | 37 | 36.5 | �Unwanted sexual contact from household member | 9 | ** |
Property crime | 2035 | 36.7 | Property crime | 995 | 41.8 |
�Something stolen out of hands or pockets | 304 | 46.2 | �Larceny | 927 | 40.7 |
�Other theft | 360 | 24.8 | � | � | � |
�Tried to steal | 203 | 11.7 | �Attempt larceny | 52 | 53.5 |
� | � | � | �Robbery | 16 | 59.6 |
�Something stolen off car | 796 | 40.0 | � | � | � |
�Bike theft | 372 | 46.2 | � | � | � |
Burglary | 719 | 59.5 | Burglary | 248 | 45.5 |
�Get in previous house to steal | 38 | 69.0 | Burglary | 194 | 45.1 |
�Get in previous house and cause damage | 10 | 79.3 | � | � | � |
�Get in house since moved in to steal | 8 | ** | � | � | � |
�Get in current house to steal | 250 | 75.7 | � | � | � |
�Get in current house and cause damage | 37 | 70.3 | � | � | � |
Try to get in previous house to steal/damage | 21 | 15.4 | Attempted burglary | � | � |
�Try to get in current house to steal/damage | 355 | 48.0 | � | 54 | 47.5 |
�Motor vehicle theft | 130 | 89.7 | Motor vehicle theft | 33 | 73.5 |
Rights and permissions
Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Pina-S�nchez, J., Buil-Gil, D., Brunton-Smith, I. et al. The Impact of Measurement Error in Regression Models Using Police Recorded Crime Rates. J Quant Criminol 39, 975–1002 (2023). https://doi.org/10.1007/s10940-022-09557-6
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10940-022-09557-6