How is this list curated?

[ColinEberhardt]

Please don't mistake this for trolling, this is a serious question!

For the skills listed in this awesome list, how do you assess the quality of the skills? How do you determine whether they deliver the functionality they claim? That it delivers quality results?

[travisvn]

@ColinEberhardt

It's challenging.

Most skill submissions are automated, and most skills are created with Claude Code to begin with.

Reviewing PRs has become more challenging in the age of generative AI coding assistance, as it can take much longer to verify the functionality and validity than it might to have AI write the code in the first place.

This is something plenty in the open-source community have had to contend with, and it's especially present for this repo.

In lieu of a lengthy response, the general heuristic is manual evaluation with layers of filtering before the full manual evaluation.

how do you assess the quality of the skills?

1. Comparing what the skill does relative to what current generative AI coding capabilities can perform at baseline
2. Assessing the "value-add" of the skill relative to the field or area it pertains to
3. Using "social proof" as a general guide (rule of thumb being GitHub stars as a baseline)

How do you determine whether they deliver the functionality they claim? That it delivers quality results?

There are zero guarantees with this awesome list.

And everyone should exercise caution when adding skills (or any plugins) to Claude Code, as the security vulnerabilities can be significant.

Often times, I'll wait to see if a certain repository picks up traction (gets stars over time) and see if it grows / what the community around it becomes. If it clearly has users and people are benefitting from using the skill, then it will be up for consideration for being added to the list here.

It's also strongly encouraged that anyone who sees issues (with functionality or security risks) to mention them here in the discussions or to create an issue (depending on severity).

It's an imperfect system, and I lean heavily towards the side of precaution. I'd rather have a shorter "awesome" list with quality options than a lengthy list that includes everything under the sun and gives no indication of value.

[ColinEberhardt]

Thank you for your detailed and considered response - really appreciate it.

Most skill submissions are automated, and most skills are created with Claude Code to begin with.

I have a similar workflow, I ask Claude (or some other AI) to draft a skill, which always make me feel a little uneasy!

manual evaluation with layers of filtering before the full manual evaluation

That feels like a pragmatic solution, and honestly, its more than other folk seem to be doing. Far too many people are following the hype first, fact later approach.

I think we're walking into a new world of agentic systems where our standard software development toolkit for creating quality systems no longer applies.

It's an imperfect system, and I lean heavily towards the side of precaution. I'd rather have a shorter "awesome" list with quality options than a lengthy list that includes everything under the sun and gives no indication of value.

Again, kudos on this approach. I've created one or two awesome lists and consider curation to be one of the most important aspects.

Keep up the great work.