Harbor: A framework for evaluating and optimizing agents and models in container environments
Year: 2026
Type: software
Abstract: What's Changed Add per-agent concurrency limits and agent-end hooks by @alexgshaw in https://github.com/harbor-framework/harbor/pull/2014 Add Blaxel network policy and DinD support by @mstolarzblaxelai in https://github.com/harbor-framework/harbor/pull/1980 feat(agents): propagate AgentConfig.env to all agent load paths by @EazyReal in https://github.com/harbor-framework/harbor/pull/1967 feat: mvp for run launcher directly in viewer by @kobe0938 in https://github.com/harbor-framework/harbor/pull/2017 feat: implement git repo skills by @LakshyAAAgrawal in https://github.com/harbor-framework/harbor/pull/1909 Skip uv download if already installed for mini swe agent by @milo-modal in https://github.com/harbor-framework/harbor/pull/1952 Remove dead Claude Agent SDK usage by @alexgshaw in https://github.com/harbor-framework/harbor/pull/2023 feat(atif): bring installed-agent trajectory converters to full ATIF v1.7 fidelity by @AlienKevin in https://github.com/harbor-framework/harbor/pull/1888 Add EC2 sandbox environment by @keuw in https://github.com/harbor-framework/harbor/pull/2031 [CI] Post adapter validation as a fresh timestamped comment by @crystalxyz in https://github.com/harbor-framework/harbor/pull/2040 Handle provider API usage limit errors by @alexgshaw in https://github.com/harbor-framework/harbor/pull/2037 add harbor run --install-only flag by @jkunstle in https://github.com/harbor-framework/harbor/pull/1983 ci: update to windows server 2025 by @xiaoxiangmoe in https://github.com/harbor-framework/harbor/pull/2033 fix(swe-agent): don't let testbed-conda.sh abort under set -u by @richardzhuang0412 in https://github.com/harbor-framework/harbor/pull/2039 feat: combine --agent-import-path into --agent by @kobe0938 in https://github.com/harbor-framework/harbor/pull/1988 test(pass-at-k): add unit tests for pass@k metric utility by @xyuzh in https://github.com/harbor-framework/harbor/pull/2046 fix(docker): fall back to tar upload when compose cp fails by @mattheliu in https://github.com/harbor-framework/harbor/pull/1944 feat(cli): add agent/verifier log-filter flags to harbor trial start by @xyuzh in https://github.com/harbor-framework/harbor/pull/2047 fix(openhands-sdk): use uv to pin Python >=3.12 for SDK venv by @neubig in https://github.com/harbor-framework/harbor/pull/2041 Api key auth by @scvance in https://github.com/harbor-framework/harbor/pull/2062 Network mode support for docker environment by @xiaoxiangmoe in https://github.com/harbor-framework/harbor/pull/1799 Preview task definition image files by @prabal-mercor in https://github.com/harbor-framework/harbor/pull/2055 Add bash action, start_url, and configurable IO dir to computer-1 by @RishiDesai in https://github.com/harbor-framework/harbor/pull/2042 fix(resume): tolerate empty/truncated result.json by @mvanhorn in https://github.com/harbor-framework/harbor/pull/2053 fix(cwsandbox): Default sandbox egress to public network mode by @matthoare117-wandb in https://github.com/harbor-framework/harbor/pull/2066 Fix Gemini CLI JSONL session handling by @RishiDesai in https://github.com/harbor-framework/harbor/pull/2069 Cap Anthropic cache_control breakpoints at 4 by @RishiDesai in https://github.com/harbor-framework/harbor/pull/2072 Harden Kimi CLI runtime configuration by @RishiDesai in https://github.com/harbor-framework/harbor/pull/2070 deps: pin transformers>=5.3.0 to fix CVE-2026-4372 by @donald-pinckney in https://github.com/harbor-framework/harbor/pull/2004 Fix mini-swe-agent ATIF steps for responses API by @henryre in https://github.com/harbor-framework/harbor/pull/2073 feat(launcher): multi-agent/multi-dataset run launcher w/ configs by @kobe0938 in https://github.com/harbor-framework/harbor/pull/2057 feat(env): add Cua Cloud environment for desktop VMs by @ddupont808 in https://github.com/harbor-framework/harbor/pull/1957 Add Codex web search CLI flag by @alexgshaw in https://github.com/harbor-framework/harbor/pull/2032 Reorganize example configs by category by @alexgshaw in https://github.com/harbor-framework/harbor/pull/2083 Add max_tokens support to MiniSweAgent by @alexgshaw in https://github.com/harbor-framework/harbor/pull/2082 Refine trial viewer typography and step detail interactions by @alexgshaw in https://github.com/harbor-framework/harbor/pull/2077 fix(langgraph): forward task-environment MCP servers to the graph by @nick-hollon-lc in https://github.com/harbor-framework/harbor/pull/2085 feat(agents): skip install when claude-code is already present + guard mkdir by @astefanutti in https://github.com/harbor-framework/harbor/pull/1970 feat: reload past run configs by @kobe0938 in https://github.com/harbor-framework/harbor/pull/2076 Link observation results to source tool calls by @alexgshaw in https://github.com/harbor-framework/harbor/pull/2087 feat(viewer): save launcher config as YAML by @kobe0938 in https://github.com/harbor-framework/harbor/pull/2089 Remove job plugins from config files by @alexgshaw in https://github.com/harbor-framework/harbor/pull/2074 Move verifier job options into a dedicated help panel by @alexgshaw in https://github.com/harbor-framework/harbor/pull/2019 feat(modal): add dynamic network policy switching by @cnnradams in https://github.com/harbor-framework/harbor/pull/2003 feat(modal): Toggle modal dynamic network configuration based on phase_network_policies by @cnnradams in https://github.com/harbor-framework/harbor/pull/2092 feat(rewardkit): opt-in cross-dimension aggregation for reward.json by @Kurry in https://github.com/harbor-framework/harbor/pull/2010 feat: launcher browse dataset path by @kobe0938 in https://github.com/harbor-framework/harbor/pull/2093 Fix Credential Deletion by @scvance in https://github.com/harbor-framework/harbor/pull/2098 [codex] Remove dead code paths by @alexgshaw in https://github.com/harbor-framework/harbor/pull/2086 rewardkit: add rubric metadata controls by @Kurry in https://github.com/harbor-framework/harbor/pull/1847 fix(docker): apply cpu/memory limits under docker compose by @mvanhorn in https://github.com/harbor-framework/harbor/pull/2051 Harbor Hub CLI Integration by @scvance in https://github.com/harbor-framework/harbor/pull/2088 fix(langgraph): respect UV_PRERELEASE env var for dependency installs by @nick-hollon-lc in https://github.com/harbor-framework/harbor/pull/2064 fix(langsmith): report token usage so experiment token columns populate by @Nickbohm555 in https://github.com/harbor-framework/harbor/pull/2061 feat(langgraph): auto-forward tracing and provider env vars to agent container by @nick-hollon-lc in https://github.com/harbor-framework/harbor/pull/2063 fix(terminus_2): emit ATIF context_management on summarization handoff step by @easyrider11 in https://github.com/harbor-framework/harbor/pull/1844 feat(rewardkit): register MCP servers for agent judges via mcp add by @benediktstroebl in https://github.com/harbor-framework/harbor/pull/2008 feat: id refactoring by @kobe0938 in https://github.com/harbor-framework/harbor/pull/2091 feat: model-name autocomplete from litellm by @kobe0938 in https://github.com/harbor-framework/harbor/pull/2101 Paginated trial downloading for hosted jobs by @scvance in https://github.com/harbor-framework/harbor/pull/2100 New Contributors @milo-modal made their first contribution in https://github.com/harbor-framework/harbor/pull/1952 @keuw made their first contribution in https://github.com/harbor-framework/harbor/pull/2031 @xyuzh made their first contribution in https://github.com/harbor-framework/harbor/pull/2046 @mattheliu made their first contribution in https://github.com/harbor-framework/harbor/pull/1944 @prabal-mercor made their first contribution in https://github.com/harbor-framework/harbor/pull/2055 @mvanhorn made their first contribution in https://github.com/harbor-framework/harbor/pull/2053 @donald-pinckney made their first contribution in https://github.com/harbor-framework/harbor/pull/2004 @ddupont808 made their first contribution in https://github.com/harbor-framework/harbor/pull/1957 @astefanutti made their first contribution in https://github.com/harbor-framework/harbor/pull/1970 @Nickbohm555 made their first contribution in https://github.com/harbor-framework/harbor/pull/2061 @easyrider11 made their first contribution in https://github.com/harbor-framework/harbor/pull/1844 Full Changelog: https://github.com/harbor-framework/harbor/compare/v0.15.0...v0.16.0
Author: Harbor Framework Team
Language: English
Subfield: Information Systems and Management
Field: Decision Sciences
Domain: Social Sciences
Keywords: ports, agent evaluation
Open Access status: green
OpenAlex ID: W7166318274
