PhD, HKUST | Expected Graduation: June 2027
Target: LLM Agent / RL / Agent Systems

Top Highlights

• SeekBench — benchmark for epistemic competence in search agents. ICLR 2026.
• FoldAct — long-horizon optimization for LLM agents with up to 5.19x training speedup. GitHub
• ByteDance system — controllable long-running agent system with autonomous execution, iterative refinement, and human interruption.

Education

Hong Kong University of Science and Technology (2023 Fall -)
Doctor of Philosophy (PhD) in Electronic and Computer Engineering

My supervisor: Prof. Wei Zhang (HKUST) (Mentor: Prof. Bing Luo (DKU))

The Chinese University of Hong Kong, Shenzhen (2019 — 2023)
Bachelor of Engineering in Electrical and Computer Engineer, Stream: Computer Engineering

Representative Research

SeekBench: Benchmarking Epistemic Competence in LLM Search Agents

• First Author | Benchmark Design & Evaluation Methodology | ICLR 2026
• Developed SeekBench, a standardized benchmark for evaluating the epistemic competence of LLM search agents beyond end-task accuracy.
• Defined a trajectory-level evaluation paradigm for how agents gather, revise, and calibrate evidence during search.
• Introduced three core metrics: Groundedness, Recovery, and Calibration. GitHub

FoldAct: Efficient and Stable Context Folding for Long-Horizon Search Agents

• First Author | Algorithm Design | arXiv preprint, 2025
• Proposed FoldAct, a context-folding algorithm for long-horizon LLM agents under multi-turn reinforcement learning.
• Developed a structured training strategy that stabilizes summary-policy learning and improves optimization efficiency.
• Achieved up to 5.19x training speedup on complex search-agent tasks while maintaining strong long-horizon decision quality. GitHub

MorphAgent: Self-Evolving Multi-Agent Collaboration Platform

• Co-first Author | System Architecture & Adaptive Collaboration | ICML-MAS 2025
• Designed a decentralized collaboration framework in which LLM agents dynamically evolve roles without predefined structures.
• Demonstrated improved task performance, transferability, and robustness across reasoning and coding benchmarks.

Beyond Right to be Forgotten: Managing Heterogeneity Side Effects Through Strategic Incentives

• First Author | Incentive Mechanism Design & Theoretical Analysis | ACM MobiHoc 2025
• Studied heterogeneity side effects in federated unlearning under non-IID data.
• Developed a Stackelberg-game-based incentive mechanism to retain crucial clients and improve stability and efficiency.

Publications

1. Shao, J., Lin, Y., Lohani, M. P., Miao, Y., and Luo, B., "Do LLM Agents Know How to Ground, Recover, and Assess? A Benchmark for Epistemic Competence in Information-Seeking Agents", ICLR 2026, 2026. (Accepted) 🎉 [arXiv]
2. Shao, J., Lin, T., Xiaojin Zhang Qiang Yang, and Luo, B., "Beyond Right to be Forgotten: Managing Heterogeneity Side Effects Through Strategic Incentives", ACM MobiHoc 2025, 2025. (Accepted) 🎉
3. Lu, S.*, Shao, J.*, Luo, B., and Lin, T., "Morphagent: Empowering agents through self-evolving profiles and decentralized collaboration", ICML-MAS, 2025. (*Equal contribution)
4. Shao, J.,Yuan, T., Lin, T., and Luo, B., Cognitive Insights and Stable Coalition Matching for Fostering Multi-Agent Cooperation, arXiv e-prints, arXiv:2405.18044.
5. Fan, T., Gu, H., Cao, X., Chan, C. S., Chen, Q., Chen, Y., Feng, Y., Gu, Y., Geng, J., Luo, B., Liu, S., Ong, W. K., Ren, C., Shao, J., Sun, C., Tang, X., Tae, H. X., Tong, Y., Wei, S., Wu, F., Xi, W., Xu, M., Yang, H., Yang, X., Yan, J., Yu, H., Yu, H., Zhang, T., Zhang, Y., Zhang, X., Zheng, Z., Fan, L., and Yang, Q., "Ten challenging problems in federated foundation models", IEEE Transactions on Knowledge and Data Engineering, 2025.
6. He, S., Tang, B., Zhang, B., Shao, J., Ouyang, X., Nugraha, D. N., and Luo, B., "Fedkit: Enabling cross-platform federated learning for android and ios", in IEEE INFOCOM 2024-IEEE conference on computer communications workshops (INFOCOM WKSHPS), 2024.
7. Geng, J., Tang, B., Zhang, B., Shao, J., and Luo, B., "FedCampus: A Real-world Privacy-preserving Mobile Application for Smart Campus via Federated Learning & Analytics", in Proceedings of the Twenty-Fifth International Symposium on Theory, Algorithmic Foundations, and Protocol Design for Mobile Networks and Mobile Computing, 2024.
8. Shao, J., Han, S., He, C., and Luo, B., "Privacy-Preserving Federated Heavy Hitter Analytics for Non-IID Data", in Workshop on Federated Learning and Analytics in Practice: Algorithms, Systems, Applications, and Opportunities, in Conjunction with ICML 2023 (FL-ICML' 23), Jul. 2023.

Experience

ByteDance | Intern (Agent Long-Horizon Self-Iterative Algorithm Systems)

• Independently led end-to-end implementation of a long-running agent and self-iterative algorithm project, from design to deployment.
• Designed and implemented a Daemon + Rubric + Harness architecture for daemonized scheduling/state hosting, rubric-driven evaluation/iteration, and harness-based orchestration/replay validation.
• Enabled Auto / Interactive / Human-Interrupt modes to support autonomous execution, collaborative workflows, and manual takeover.
• Built robust state management (stage transitions, failure recovery, and human handoff) to improve stability and controllability in long-horizon tasks.

Projects

MASArena: Benchmarking Framework for Multi-Agent Systems

• Led the design and implementation of MASArena, an open-source, modular benchmarking framework for single- and multi-agent systems, co-developed by DKU-Edge Intelligence Lab and Westlake University LINs-Lab.
• Architected the core framework with plug-and-play modules, built-in benchmarks, visual debugging capabilities, and seamless agent/tool/dataset integration.
• Implemented key components including experiment orchestration, agent lifecycle management, and real-time monitoring infrastructure.
• Designed the evaluation pipeline to support academic experiment reproduction, agent comparison, and toolchain evaluation with comprehensive metrics collection.
• Open-source and actively maintained. GitHub link

FedKit: Enabling Cross-Platform Federated Learning for Android and iOS

• We present FEDKIT, which pipelines Cross-Platform FL for Android and iOS development by enabling model conversion, hardware-accelerated training, and cross-platform model aggregation.
• Our workflow supports flexible federated learning operations (FLOps) in production, facilitating continuous model delivery and training.
• This is a collaborative project with Prof. Luo, DKU undergraduate students Sichang He (lead), Beilong Tang, and Boyan Zhang, as well as collaborators Xiaomin Ouyang (UCLA) and Daniel Nata (Flower).
• Our work has been ACCEPTED at IEEE INFOCOM 2024 Demo 🎉.

FedKit Pipeline Overview	FedKit Simulation

FedCampus: A Privacy-Preserving Data Platform for Smart Campus

• We’re excited to announce the launch of the FedCampus Project - a privacy-preserving smart campus application, available on Android and iOS. 🎉 Video online available.
• This app implements two key privacy-preserving technologies: Federated Learning and Differential Privacy. Check out our 100 customized smart watches for participants at DKU and FedCampus APP.
• This is a collaborative project with Prof. Luo and DKU undergraduate students.

Edge-based Cross-device Federated Learning Prototypes

• Our prototype supports Mobile and IoT devices operating at WiFi and USRP-based 4G/5G wireless networks.
• This is a collaborative project with Prof. Luo and students from CUHKSZ

Talks & Invited Seminars

• Vibe Coding Seminar, Duke Kunshan University (DKU), April 2025. Invited seminar.

Teaching Assistant

• ELEC3120: Computer Communication Networks (HKUST, Spring 2024)
• ELEC3300: Introduction to Embedded Systems (HKUST, Fall 2024)
• Vector Space Methods with Applications | ECE 586K (DKU, Spring 2025)
• Advanced Topics in Electrical and Computer Engineering | ECE 590K (DKU)

Patents

• B. Luo, J. Shao, Method and Apparatus for Online Parameter Selection in Minimizing the Total Cost of Federated Learning, CN202310485067.8, Apr. 2023, field
• B. Luo, J. Shao, Method and Apparatus for Online Client Sampling in Minimizing the Training time of Federated Learning, CN 202310484383.3, Apr. 2023, field
• B. Luo, J. Shao, J. Huang, Method and Apparatus for Frequent Items Mining Using Federated Analytics, CN202310365167.7, Mar. 2023, field
• B. Luo, J. Shao, J. Huang, Method and Apparatus for Frequent Data Mining Based on Hierarchical Federated Analytics, CN202310330791.3, Mar. 2023, field