LLM assistants increasingly retain long-term memories of their users, but retaining a memory is not the same as knowing when to use it. We introduce PersistBench, a benchmark for two safety risks that emerge from persistent memory: cross-domain leakage, where models inappropriately inject stored personal context into unrelated requests, and memory-induced sycophancy, where retained memories cause models to reinforce a user’s stated beliefs. Evaluating 18 models, we find a median failure rate of 53% on cross-domain samples and 97% on sycophancy samples, indicating that current memory-augmented assistants lack reliable mechanisms for deciding when a memory should be forgotten.
@inproceedings{pulipaka2026persistbench,title={PersistBench: When Should Long-Term Memories Be Forgotten by LLMs?},author={Pulipaka, Sidharth and Chen, Oliver and Sharma, Manas and Bajwa, Taaha S. and Raina, Vyas and Sheth, Ivaxi},booktitle={International Conference on Machine Learning (ICML)},year={2026},note={*Equal contribution.}}
This work examines how the May 2025 India-Pakistan conflict reshaped domestic news coverage in Pakistan. Using LLMs to analyse roughly 2,600 articles from three major newspapers, it finds that war-related reporting significantly overshadowed coverage of political opposition and dissent. The results suggest that regional conflict can suppress democratic dialogue, underscoring the importance of protecting journalistic independence in politically unstable regions.
@inproceedings{bajwa2025wartime,title={Wartime Media Dynamics in Emerging Democracies: Case Study of Pakistani Media in May 2025 Indo-Pak Conflict},author={Bajwa, Taaha Saleem},booktitle={NLP4Democracy Workshop, Conference on Language Modeling (COLM)},year={2025},note={Extended abstract, presented as a poster.}}
An evaluation of how large language models handle Urdu, a low-resource language, separating comprehension from generation. The work finds that LLMs primarily reason in English and use their final layers as a translator into the target language: the internal English reasoning is coherent, but the translation step introduces significant information loss, so models comprehend Urdu substantially better than they generate it. Mechanistically ablating these translation features and applying a dedicated translation model to the internal representations improves performance while better preserving cultural context.
@inproceedings{bajwa2025understanding,title={Understanding vs. Generation: LLMs Are Better at Comprehending Low-Resource Languages like Urdu Than Generating Text in Them},author={Bajwa, Taaha Saleem},booktitle={International Conference on Social Impact of AI (SIAI-ReDI), AAAI 2025 Workshop},pages={107--111},year={2025},note={Extended abstract, presented as a poster.}}
