How to Find a Needle in a Haystack
Searching for a needle in a haystack is an important task in several contexts of data analysis and decision-making. Examples include identifying the insider threat within an organization, the prediction of failure in industrial production, or pinpointing the unique signature of a solo perpetrator, such as a school shooter or a lone wolf terrorist. It is a challenge different from that of identifying a rare event (e.g., a tsunami) or detecting anomalies because the "needle" is not easily distinguished from the haystack. This challenging context is imbued with particular difficulties, from the lack of sufficient data to train a machine learning model through the identification of the relevant features and up to the painful price of false alarms, which might cause us to question the relevance of machine learning solutions even if they perform well according to common performance criteria. In this book, Prof. Neuman approaches the problem of finding the needle by specifically focusing on the human factor, from solo perpetrators to insider threats. Providing for the first time a deep, critical, multidimensional, and methodological analysis of the challenge, the book offers data scientists and decision makers a deep scientific foundational approach combined with a pragmatic practical approach that may guide them in searching for a needle in a haystack.
Yair Neuman is a full professor at Ben-Gurion University of the Negev. He is the author of numerous papers and eight books published by leading academic publishers from Cambridge University Press to Brill and Springer Nature. He is consistently ranked within the top 3% of researchers on Academia.edu (https://bgu.academia.edu/YairNeuman).
Prof. Neuman’s computational and data analytics projects have been supported by government agencies (e.g. IARPA – the Intelligence Advanced Research Projects Activity) and world-leading banks, and he has served as a scientific advisor to various clients in the private sector. His novel data analysis methodologies have been published in leading journals and cover human, industrial, medical, and financial data.