Danica M. Ommen

The online supplement provides additional figures described in this article and the details of non-Gaussian distributions used in the simulation study.

A variety of statistical approaches have been developed at the Center for Statistics and Applications in Forensic Evidence (CSAFE) to address the question of writership for forensic document examinations. Previous work at CSAFE has addressed the closed-set problem, when the writer of a questioned document must be one out of a list of known writers. This presentation focuses on a set of evidence interpretation methods that can address the open-set problem, when the writer of a questioned document could be an unknown writer not included in the known list. To date, CSAFE has developed three types of statistical writership evaluation methods, a Two-Stage approach, a score-based likelihood ratio, and a Bayes Factor. All methods are demonstrated on a set of handwriting collected by CSAFE and features of the writing are extracted using the CSAFE-developed ‘handwriter’R package. The key components of …

This Supplementary Material contains the code used to produce all of the results presented in this manuscript. The glass fragment data set that we investigate (van Es et al. (2017)) was kindly supplied by the NFI, but the NFI was not further involved in this research. Currently, these data are not publicly available, but are available on request by emailing p.zoon@nfi.nl.

Questioned Document Examiners (QDEs) are tasked with analyzing handwriting evidence to make source (or writership) determinations. The Center for Statistics and Applications of Forensic Evidence (CSAFE) has previously developed computational methods to automatically extract quantifiable handwriting features and statistical methods to analyze handwriting evidence to aid QDEs. 1-3 The method developed by Crawford et. al uses a K-means clustering algorithm and Bayesian hierarchical model to perform closed-set writer identification. 2 This means a questioned document is assigned to its most likely writer from a set of known writers but does not allow for the possibility of the questioned document to be written by someone not included in the set. Another method developed by Johnson and Ommen utilized machine learning techniques and score-based likelihood ratios (SLRs). 3 SLRs have been criticized …

Camera identification addresses the scenario where an investigator has a questioned digital image from an unknown camera. The investigator wants to know whether the questioned image was taken by a camera on a person of interest’s phone. Researchers discovered that slight imperfections in a camera’s sensor array can be used as an identifying feature or camera fingerprint in this scenario. These imperfections result from the manufacturing process and cause some pixels to produce consistently brighter or darker values than their neighboring pixels. A camera leaves its fingerprint in the images that it takes. A camera fingerprint is typically estimated from 50 or more reference images known to have been taken by the person of interest’s camera. A camera fingerprint is also estimated from the questioned image. Most camera identification methods measure the similarity between the two camera fingerprints with a …

Machine learning‐based score likelihood ratios (SLRs) have emerged as alternatives to traditional likelihood ratios and Bayes factors to quantify the value of evidence when contrasting two opposing propositions. When developing a conventional statistical model is infeasible, machine learning can be used to construct a (dis)similarity score for complex data and estimate the ratio of the conditional distributions of the scores. Under the common source problem, the opposing propositions address if two items come from the same source. To develop their SLRs, practitioners create datasets using pairwise comparisons from a background population sample. These comparisons result in a complex dependence structure that violates the independence assumption made by many popular methods. We propose a resampling step to remedy this lack of independence and an ensemble approach to enhance the performance …

Trained experts currently perform the handwriting analysis required in the criminal justice field, but this can create biases, delays, and expenses, leaving room for improvement. Prior research has sought to address this by analyzing handwriting through feature-based and score-based likelihood ratios for assessing evidence within a probabilistic framework. However, error rates are not well defined within this framework, making it difficult to evaluate the method and can lead to making a greater-than-expected number of errors when applying the approach. This research explores a method for assessing handwriting within the Two-Stage framework, which allows for quantifying error rates as recommended by a federal report by PCAST (Forensic Science in Criminal Courts: Ensuring Scientific Validity of Feature Comparison Methods). The coincidence probabilities produced here can be used in later research to asses …

Machine learning-based Score Likelihood Ratios have been proposed as an alternative to traditional Likelihood Ratios and Bayes Factor to quantify the value of evidence when contrasting two opposing propositions.

We develop a statistical approach to model handwriting that accommodates all styles of writing (cursive, print, connected print). The goal is to compute a posterior probability of writership of a questioned document given a closed set of candidate writers. Such probabilistic statements can support examiner conclusions and enable a quantitative forensic evaluation of handwritten documents. Writing is treated as a sequence of disjoint graphical structures, which are extracted using an automated and open‐source process. The graphs are grouped based on the similarity of their shapes through a K‐means clustering template. A person's writing pattern can be characterized by the rate at which graphs are emitted to each cluster. The cluster memberships serve as data for a Bayesian hierarchical model with a mixture component. The rate of mixing between two parameters in the hierarchy indicates writing style.

Machine learning-based Score Likelihood Ratios have been proposed as an alternative to traditional Likelihood Ratios and Bayes Factor to quantify the value of evidence when contrasting two opposing propositions.

Identifying the source of handwriting is an important application in the field of forensic science that addresses questioned document evidence found in criminal cases and civil litigation. It is difficult, given the idiosyncrasies of a person’s handwriting, to recognize the exact writer of a piece of handwriting based only on its physical properties. Even more so is trying to classify a writer without any prior database containing handwriting characteristics of such writer. Data sets containing handwriting samples from different sources are used to investigate how well a convolutional neural network can classify writers from unseen sources. Comparisons between scenarios modeled after real-world situations with varying degrees of complexity, which are adjusted by whether and from which source the samples from the suspects have been collected to train the model, are made to examine the extent to which twin convolutional …

An overview of source camera identification on multi-camera smartphones, and introduction to the new CSAFE multi-camera smartphone image database, and a summary of recent results on the iPhone 14 Pro's.

Aluminum (Al) powder is often used as a fuel in explosive devices; therefore, individuals attempting to make illegal improvised explosive devices often obtain it from legitimate commercial products or make it themselves using readily available Al starting materials. The characterization and differentiation between sources of Al powder for additional investigative and intelligence value has become increasingly important. Previous research modeled the distributions of micromorphometric features of Al powder particles within a subsample to support Al source discrimination. Since then, additional powder samples from a variety of different source types have been obtained and analyzed, providing a more comprehensive dataset for applying the two statistical methods for interpretation and discrimination of source. Here, we compare two different statistical techniques: one using linear discriminant analysis (LDA), and the other using a modification to the method used in ASTM E2927-16e1 and E2330-19. The LDA method results in an Al source classification for each questioned sample. Alternatively, our modification to the ASTM method uses an interval-based match criterion to associate or exclude each of the known sources as the actual source of a trace. Although the outcomes of these two statistical methods are fundamentally different, their performance with respect to the closed-set identification of source problem is compared. Additionally, the modified ASTM method will be adapted to provide a vector of scores in lieu of the binary decision as the first step towards a score-based likelihood ratio for interpreting Al powder micromorphometric …

Handwriting comparative analysis is based on the principle that no two individuals can produce the same writing and that an individual cannot exactly reproduce his/her handwriting. This project aims to assess and quantify the natural variations produced by a distinct writer. In an attempt to support traditional examination with objective measures, this project provides results from a study where features of handwriting are examined through graph decomposition and rainbow triangulation. Using this method to examine handwriting samples, more specific information can be obtained from each exemplar and can be standardized to be compared both within a writer and between different writers. Each type of characteristic or landmark of each handwriting sample are marked as a different color node in a graph, including the location that a pen stroke begins (blue), the location that a pen stroke ends (orange), any location …

Appendix A contains details regarding a method of selecting potentially discriminating clusters, and Appendix B contains details of the prior sensitivity analysis for the final hierarchical model.

Forensic camera device identification addresses the scenario, where an investigator has two pieces of evidence: a digital image from an unknown camera involved in a crime, such as child pornography, and a person of interest’s (POI’s) camera. The investigator wants to determine whether the image was taken by the POI’s camera. Small manufacturing imperfections in the photodiode cause slight variations among pixels in the camera sensor array. These spatial variations, called photo‐response non‐uniformity (PRNU), provide an identifying characteristic, or fingerprint, of the camera. Most work in camera device identification leverages the PRNU of the questioned image and the POI’s camera to make a yes‐or‐no decision. As in other areas of forensics, there is a need to introduce statistical and probabilistic methods that quantify the strength of evidence in favor of the decision. Score‐based likelihood ratios (SLRs …

Machine learning-based Score Likelihood Ratios have been proposed as an alternative to traditional Likelihood Ratio and Bayes Factor to quantify the value of forensic evidence. Scores allow formulating comparisons using a lower-dimensional metric, which becomes relevant for complex evidence where developing a statistical model becomes challenging. Under this framework, a (dis) similarity score and its distribution under alternative propositions is estimated using pairwise comparison from a sample of the background population. These procedures often rely on the independence assumption, which is not met when the database consists of pairwise comparisons. To remedy this lack of independence, we introduce an ensembling approach that constructs training and estimation sets by sampling forensic sources, ensuring they are selected only once per set. Using these newly created datasets, we construct …

Identifying the source of handwriting is an important application in the field of forensic science that addresses questioned document evidence found in criminal cases and civil litigation. It is difficult, given the idiosyncrasies of a person's handwriting, to recognize the exact writer of a piece of handwriting based only on its physical properties. Even more so, trying to classify a writer without any prior database containing handwriting characteristics of such writer. data sets containing handwriting samples from different sources are used to investigate how well a convolutional neural network can classify writers from unseen sources. This paper aims to compare data processing and modeling methods to improve classification on whether two pieces of handwriting are from the same or different writer, in the context where every potential writer has never been seen before. The structure of a twin convolutional neural network …

Advances in the interpretation of forensic evidence have led to a number of different statistical methods all reaching for a quantification of the evidence, but using different techniques within different facets of statistical science. Many of these methods have been lumped into what is typically referred to as “likelihood ratio approaches.” This had caused great confusion within the field of forensic sciences on what to call the methods commonly used in practice and which methods to consider using in the future. The overall goal of this chapter is to provide a straightforward account of the similarities and differences between a Bayes Factor and a likelihood ratio for the forensic identification of source problems.

An investigator wants to know if an illicit image captured by an unknown camera was taken by a person of interest’s (POI’s) phone. Score-based likelihood ratios (SLRs) have been used to answer this question in previous research. We explore whether the reference set used to calculate SLRs makes a difference in the outcome when the questioned image comes from a phone of the same model as the POI’s phone.