Yiqiao Jin CS PhD @ Georgia Tech
  • Bio
  • Papers
  • Talks
  • News
  • Experience
  • Projects
  • Teaching
  • Publications
    • MASCOT: Towards Multi-Agent Socio-Collaborative Companion Systems
    • MM-BizRAG: Rethinking Multimodal Retrieval-Augmented Generation for General Purpose Enterprise Q&A
    • Reasoning Is Not All You Need: Examining LLMs for Multi-Turn Mental Health Conversations
    • SARA: Selective and Adaptive Retrieval-augmented Generation with Context Compression
    • MedHalu: Hallucinations in Responses to Healthcare Queries by Large Language Models
    • TextReg: Mitigating Prompt Distributional Overfitting via Regularized Text-Space Optimization
    • UniSD: Towards a Unified Self-Distillation Framework for Large Language Models
    • Beyond Magic Words: Sharpness-Aware Prompt Evolving for Robust Large Language Models
    • Sysformer: Safeguarding Frozen Large Language Models with Adaptive System Prompts
    • AgentArk: Distilling Multi-Agent Intelligence into a Single LLM Agent
    • Consistency Should Be the Priority for Unified Multimodal Models
    • Efficient Knowledge Probing of Large Language Models by Adapting Pre-trained Embeddings
    • SlideAgent: Hierarchical Agentic Framework for Multi-Page Visual Document Understanding
    • Protein Large Language Models: A Comprehensive Survey
    • Empowering Interdisciplinary Insights with Dynamic Graph Embedding Trajectories
    • Deconstructing The Ethics of Large Language Models from Long-standing Issues to New-emerging Dilemmas
    • A Survey on Efficient LLM Training: From Data-centric Perspectives
    • ProteinGPT: Multimodal LLM for Protein Property Prediction and Structure Understanding
    • Topological Structure Learning Should Be A Research Priority for LLM-Based Multi-Agent Systems
    • CultureVLM: Characterizing and Improving Cultural Understanding of Vision-Language Models for over 100 Countries
    • ScreenLLM: Stateful Screen Schema for Efficient Action Understanding and Prediction
    • SciEvo: A 2 Million, 30-Year Cross-disciplinary Dataset for Temporal Scientometric Analysis
    • UniGuard: Towards Universal Safety Guardrails for Jailbreak Attacks on Multimodal Large Language Models
    • RNA-GPT: Multimodal Generative System for RNA Sequence Understanding
    • AgentReview: Exploring Peer Review Dynamics with LLM Agents
    • PrivacyMind: Large Language Models Can Be Contextual Privacy Protection Learners
    • Towards Fair Graph Anomaly Detection: Problem, Benchmark Datasets, and Evaluation
    • CompeteAI: Understanding the Competition Behaviors in Large Language Model-based Agents
    • Better to Ask in English: Cross-Lingual Evaluation of Large Language Models for Healthcare Queries
    • MM-SOC: Benchmarking Multimodal Large Language Models in Social Media Platforms
    • Prototypical Reward Network for Data-Efficient RLHF
    • Semi-Offline Reinforcement Learning for Optimized Text Generation
    • Code Recommendation for Open Source Project Developers
    • Predicting Information Pathways Across Online Communities
    • Prototypical Fine-Tuning: Towards Robust Performance under Varying Data Sizes
    • Reinforcement Subgraph Reasoning for Fake News Detection
    • Towards Fine-Grained Reasoning for Fake News Detection
  • Projects
    • UniSD
    • Sysformer
    • SARA
    • SlideAgent
    • AgentArk
    • ScreenLLM
    • SciEvo
    • AgentReview
    • MM-Soc
    • CompeteAI
    • XLingEval
    • INPAC
    • CODER
    • FinerFact
  • Recent & Upcoming Talks
    • SlideAgent: Hierarchical Agentic Framework for Multi-Page Visual Document Understanding
    • AgentReview: Exploring Peer Review Dynamics with LLM Agents
  • Projects
  • Experience
  • Teaching
    • Learn JavaScript
    • Learn Python
  • Blog

ScreenLLM

Apr 30, 2025 · 0 min read
Go to Project Site
Last updated on Apr 30, 2025
Multimodal LLMs GUI Understanding Agents
Yiqiao Jin
Authors
Yiqiao Jin
Ph.D. Candidate in Computer Science
My research focuses on adaptive and efficient AI systems, with emphasis on LLM agents, agent memory, self-distillation, multimodal LLMs, and structured multi-agent intelligence.
← AgentArk Feb 4, 2026
SciEvo Mar 3, 2025 →

© 2026 Yiqiao Jin. This work is licensed under CC BY NC ND 4.0

Published with Hugo Blox Builder — the free, open source website builder that empowers creators.