3. SUMMARY OF THE DISCUSSED RESEARCH TOPICS FOR FUTURE EXPLORATION
Here we present a brief summary of the key observations, challenges, and opportunities discussed in each breakout session. More details on each topic are provided in the following sections.
3.1. Evaluation
- Exploring the limits of using LLMs to label material as relevant or not, including automatically generated explanations that indicate why something is relevant (or not).
- Expanding evaluation approaches to include the entire search and discovery process: handling multi-step processes such as conversations, measuring the accuracy of generated (rather than simply retrieved) responses, and designing approaches that measure the interactions between IR and GenAI.
- Developing and employing “digital twin” technology to enable strong and reliable simulated user evaluations. That includes using variations of the same “twin” to more broadly understand the range of human responses.
3.2. Training, feedback, and reasoning
- Developing interactive generative AI and information retrieval systems that cooperatively learn when to submit queries and what information to retrieve that augments the knowledge encoded in generative AI systems.
- Exploring various implicit and explicit feedback mechanisms to iteratively improve retrieval-enhanced generative AI systems.
- Retrieving, organizing, and synthesizing vast amounts of information using generative AI systems that can manage complex reasoning tasks.
3.3. Understanding and modeling users
- Understanding user needs and their multimodal interactions with generative AI systems for information discovery and access.
- Learning an effective model of users by representing their cognitive state, including their state of knowledge, while using generative AI systems for information discovery and access.
- Exploring privacy-preserving solutions in modeling users in generative AI systems.
3.4. Social ramifications
-
Thinking about the impact on all aspects of society continually: before, during, and after development.
-
Identifying risks, challenges, and opportunities of generative AI for information retrieval requires an interdisciplinary approach informed by socio-technical perspectives and co-developed with social science scholars, legal scholars, civil society representatives, and policy makers among others.
3.5. Personalization
- Developing efficient, personalized generative AI systems that act as digital twins to learn personal behavior and preferences of the user for personalized retrieval, recommendation, and synthesis of information.
- Developing persuasive recommender systems that not only produce accurate recommendations but also provide sufficient explanation, transparency, and justification to persuade their users.
- Developing efficient personalized generative AI systems for on-device intelligent assistance and information access.
3.6. Reducing the cost of generative IR
- Exploring the capabilities of numerous small LLMs rather than a single monolithic approach.
- Investigating the tradeoffs between different sized LLMs and different sized IR systems.
- Exploring ways to accomplish the same (or improved) capabilities with a fixed hardware footprint rather than one that is assumed to grow continuously.
- Considering the impact of new computational paradigms (e.g., quantum, bio, or neuromorphic computing) to understand their impact on GenAI and IR.
3.7. AI agents and information retrieval
- Understanding how mixed-initiative systems leverage GenAI and IR to gather information proactively for users without direct initiation.
- Exploring how agents can communicate with each other reliably, particularly in the face of current hallucination challenges.
- Developing approaches that allow GenAI to make novel and perhaps complex plans for gathering information in response to challenging questions.
- Creating evaluation frameworks for groups of agents, some of which address multimodal inputs, collaborating with each other and users to accomplish a task.
3.8. Foundation models for information access and discovery
-
Developing a task-agnostic foundation model for information access, extending human intelligence with personalized, contextual, and multimodal information and beyond.
-
Developing techniques for better incorporating user behavior, supporting multiple modalities, and generating accessible output.
-
Extending foundation models so that they can continuously learn, improve, avoid or address biases, and so on.
-
Creating non-profit regional organizations that aim to solicit computing resources to be shared among regional researchers.