BEGIN:VCALENDAR
VERSION:2.0
PRODID:Linklings LLC
BEGIN:VTIMEZONE
TZID:Europe/Stockholm
X-LIC-LOCATION:Europe/Stockholm
BEGIN:DAYLIGHT
TZOFFSETFROM:+0100
TZOFFSETTO:+0200
TZNAME:CEST
DTSTART:19700308T020000
RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=-1SU
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:+0200
TZOFFSETTO:+0100
TZNAME:CET
DTSTART:19701101T020000
RRULE:FREQ=YEARLY;BYMONTH=10;BYDAY=-1SU
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTAMP:20241120T082409Z
LOCATION:HG D 1.2
DTSTART;TZID=Europe/Stockholm:20240605T113000
DTEND;TZID=Europe/Stockholm:20240605T120000
UID:submissions.pasc-conference.org_PASC24_sess110_msa286@linklings.com
SUMMARY:Benchmarking Economic Reasoning in Artificial Intelligence Models
DESCRIPTION:Minisymposium\n\nDouglas Araujo (Bank for International Settle
 ments)\n\nA theory-informed test of reasoning in artificial intelligence (
 AI) combines three sequential steps to consider correct answers as the res
 ult of a reasoning process as opposed to luck of probabilistic word matchi
 ng. The first step is information filtering, where an AI model that reason
 s must distinguish the relevant information in a prompt from trivia. In th
 e second step, knowledge association, the AI combines implicit or explicit
  knowledge with the relevant prompt information. And finally in the third 
 step of logic attribution, a reasoning AI assigns correct logic operations
  for deducive, inducive, and other types of logic to uncover the corret an
 swer. In economic settings, the logic steps involve different levels of co
 unterfactual considerations and policy-relevant thought experiments. This 
 paper leverages insights from the large language model benchmarking litera
 ture and the social economics literature to inform the design of benchmark
 ing tests that are challenging, robust, evolving over time and informative
  about any type of reasoning shortcomings. The benchmarking process can be
  adapted to other sciences. An accompanying training dataset is available 
 to help AI developers improve reasoninig in their models, and interested u
 sers can submit proposals for material to create questions.\n\nDomain: App
 lied Social Sciences and Humanities, Computational Methods and Applied Mat
 hematics\n\nSession Chairs: Aleksandra Friedl (ifo Institut), Simon Scheid
 egger (University of Lausanne), and Yucheng Yang (University of Zurich)
END:VEVENT
END:VCALENDAR