When LLMs hit the stage, everyone was talking about prompting. The reality however is trying to impose structure on a problem, through natural language is brittle and difficult especially if there are no metrics to hold yourself against. We challenge the notion that prompting is the primary skill for engineering effective LLM applications, proposing instead a focus on practical Data Science design patterns. Drawing from experiences in our use cases within the financial economic crime domain at ABN AMRO, our session highlights the critical aspects of: decomposition and evaluation processes to mitigate against risks and failure modes of the model.

Anna Lorincz is a dedicated Data Scientist in the Detecting Financial Crime department at ABN AMRO, where she leverages her expertise in Natural Language Processing (NLP). Her journey into the world of Large Language Models (LLMs) began with her master’s thesis, which focused on evaluating generative AI in the HR domain. In her current role, Anna is passionate about identifying and implementing impactful LLM use cases that ensure these technologies are applied both responsibly and optimally. She plays a key role in the exploration and selection of projects and is actively involved in the development of these use cases. Additionally, Anna has contributed to the development and experimentation of the evaluation framework, which will be a major highlight of the upcoming talk.

Theresa Fruhwuerth is a Senior Data Scientist at ABN Amro Detecting Financial Crime, she has come to appreciate her job as the art of investigating hypothesis and progressing, bit by bit reducing uncertainty around a system that should help people to do their job more effective. For her Generative AI is a natural extension of Natural language processing and having good mental models of NLP are really what set apart a good Data Science team working with this new technology. She tries to make these concepts available to her group and others, so we can all deploy systems to production that will remain relevant even after the hype wore off and more importantly provide value to the business.