-
Notifications
You must be signed in to change notification settings - Fork 929
Description
Describe the bug
SkillInvokedEvent (type skill.invoked) is defined in the SDK's generated SessionEvents.cs with a full data model (Name, Path, Content, AllowedTools), but it is never emitted during a session — even when the agent successfully loads a skill via SkillDirectories and calls the skill tool.
Impact
The dotnet/skills eval harness relies on this event for reliable skill activation detection (MetricsCollector.ExtractSkillActivation). Without it, activation detection falls back to an unreliable "extra tools" heuristic — checking whether the skilled run used tools the baseline didn't. This heuristic produces false negatives for knowledge-only skills that don't add new tools.
Reproduction
- Create a session with
SkillDirectoriespointing to a directory with a validSKILL.md - Subscribe to all events via
session.On(...) - Send a prompt that triggers the skill (e.g., "Use the X skill to help me")
- Observe:
ToolExecutionStartEventfires for tool"skill", butSkillInvokedEventnever fires
Tested on SDK versions 0.1.26 and 0.1.30 — skillEventCount is 0 across hundreds of agent runs in both versions.
Expected behavior
SkillInvokedEvent should fire when the agent loads/invokes a skill, providing the skill's Name, Path, and Content.
Environment
- GitHub.Copilot.SDK 0.1.26 and 0.1.30
- .NET 10
- Windows 11
Minimal repro code
skills/SKILL.md:
---
name: my-skill
description: A test skill for demonstrating SkillInvokedEvent.
---
# My Skill
When invoked, respond with "Hello from my-skill!"Program.cs:
using GitHub.Copilot.SDK;
await using var client = new CopilotClient();
await using var session = await client.CreateSessionAsync(new SessionConfig
{
Model = "claude-sonnet-4.5",
SkillDirectories = ["./skills"]
});
var skillInvokedCount = 0;
var done = new TaskCompletionSource();
session.On(evt =>
{
if (evt is SkillInvokedEvent si)
{
skillInvokedCount++;
Console.WriteLine($"Skill invoked: {si.Data.Name}");
}
else if (evt is ToolExecutionStartEvent ts && ts.Data.ToolName == "skill")
{
Console.WriteLine("Skill tool called (but no SkillInvokedEvent)");
}
else if (evt is SessionIdleEvent)
{
done.SetResult();
}
});
await session.SendAsync(new MessageOptions
{
Prompt = "Use the my-skill skill to help me."
});
await done.Task;
Console.WriteLine($"SkillInvokedEvent count: {skillInvokedCount}");
// Expected: >= 1
// Actual: 0