# Agentic AI For Good (`transparent_flea/agentic-ai-explorer-developer`) Actor

Fetches the latest agentic AI tools, models, frameworks, and research from top developer sources, then delivers structured insights and examples that keep you up to date every day

- **URL**: https://apify.com/transparent\_flea/agentic-ai-explorer-developer.md
- **Developed by:** [Nimit Savant](https://apify.com/transparent_flea) (community)
- **Categories:** AI, Agents, Developer tools
- **Stats:** 8 total users, 0 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $20.00 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Agentic AI Explorer — Developer Edition

An Apify Actor that scrapes developer-focused Agentic AI sources and converts them into structured insights for developers.

### 🎯 Overview

This Actor continuously monitors and aggregates the latest developments in agentic AI from multiple sources, transforming raw data into actionable developer insights with code examples and use cases.

### 📡 Data Sources

The Actor fetches data from:

1. **GitHub** - Trending repositories related to AI, ML, LLM, and Agent frameworks
2. **HuggingFace** - Newest models and agent frameworks
3. **PapersWithCode** - New agentic AI research papers
4. **dev.to** - Developer articles and tutorials on agentic AI
5. **Medium** - Blog posts and articles about AI/ML developments
6. **LLM-Powered Internet Search** - Uses AI search APIs (Tavily, Perplexity, Serper) to find relevant content across the web

### 🚀 Features

- **Multi-source aggregation** - Collects data from 6+ sources in parallel
- **Developer-focused insights** - Transforms raw data into actionable developer information
- **Code examples** - Generates relevant code examples for each tool/framework
- **Multiple output formats** - Supports JSON (default) and Markdown output
- **Daily subscriptions** - Automatically receive fresh insights every day at your preferred time
- **Error resilience** - Never crashes on partial failures
- **Fully typed** - Complete TypeScript type safety

### 📦 Output Structure

Each scraped item produces:

```typescript
{
  toolName: string;
  source: string;
  summary: string;
  developerUseCase: string;
  codeExample?: string;
  futureOutlook?: string;
}
```

#### Markdown Format

When using Markdown output, generates readable sections:

```markdown
#### Tool Name
**Source:** GitHub
**Summary:** ...
**Developer Use Case:** ...
**Code Example:**
```

// code here

```
**Future Outlook:** ...
```

### 🔑 API Keys & Authentication

**Good news: No API keys required for core functionality!**

All default data sources work without any API keys:

- ✅ **GitHub** - Uses public API (no authentication needed)
- ✅ **HuggingFace** - Uses public API (no authentication needed)
- ✅ **PapersWithCode** - Uses public API (no authentication needed)
- ✅ **dev.to** - Uses public API (no authentication needed)
- ✅ **Medium** - Uses web scraping (no authentication needed)

**Optional API keys:**

- 🔐 **LLM Search** - Only required if you enable `enableLLMSearch: true`
  - Choose one: `TAVILY_API_KEY`, `PERPLEXITY_API_KEY`, or `SERPER_API_KEY`
  - See [LLM-Powered Internet Search](#-llm-powered-internet-search) section for setup

**Optional (for higher rate limits):**

- 🔐 **GitHub Token** - Optional GitHub personal access token for higher API rate limits (60/hour → 5000/hour)
  - Set as `GITHUB_TOKEN` environment variable (not required, Actor works fine without it)

### ⚙️ Input Parameters

- **topics** (string\[]): Topics to search for (default: `["agentic ai", "llm frameworks"]`)
- **maxItems** (number): Maximum number of items to collect per source (default: `20`)
- **outputFormat** ("json" | "markdown"): Output format (default: `"json"`)
- **includeCodeExamples** (boolean): Whether to include code examples (default: `true`)
- **subscribe** (boolean): Enable daily automatic runs with the same query (default: `false`)
- **subscriptionTimeUtc** (string): Time of day (UTC) when you want to receive daily updates, e.g. "06:30" (default: `"06:30"`)
- **subscriptionName** (string): Optional custom name for this subscription. If not provided, a stable name is derived from topics + format (default: `""`)
- **enableLLMSearch** (boolean): Enable LLM-powered internet search to find relevant content across the web (default: `false`)
- **llmSearchProvider** ("tavily" | "perplexity" | "serper"): Choose the search provider for LLM search (default: `"tavily"`)
- **tavilyApiKey** (string, secret): Your Tavily API key. Only required if `enableLLMSearch` is true and `llmSearchProvider` is "tavily"
- **perplexityApiKey** (string, secret): Your Perplexity API key. Only required if `enableLLMSearch` is true and `llmSearchProvider` is "perplexity"
- **serperApiKey** (string, secret): Your Serper API key. Only required if `enableLLMSearch` is true and `llmSearchProvider` is "serper"
- **githubToken** (string, secret, optional): Optional GitHub personal access token for higher API rate limits (60/hour → 5000/hour)

### 🔔 Daily Subscriptions

The Actor supports automatic daily runs through Apify Schedules. When you enable subscriptions:

1. **Run once with subscription enabled** - Set `subscribe: true` in your input
2. **Actor executes normally** - Fetches sources, generates insights, delivers output
3. **Schedule is created/updated** - After completion, creates or updates an Apify Schedule
4. **Daily automatic runs** - The schedule runs the actor daily at your specified UTC time
5. **Same query, fresh data** - Each scheduled run uses the same input parameters (except `subscribe` is set to `false` to prevent recursion)

#### Subscription Features

- **Stable schedule names** - Re-running with the same subscription updates the existing schedule instead of creating duplicates
- **Custom scheduling** - Set your preferred UTC time for daily updates
- **Automatic delivery** - Output is delivered via your configured channels (email/Slack/Discord/webhook/etc.) on each scheduled run
- **No recursion** - Scheduled runs automatically have `subscribe: false` to prevent creating schedules of schedules

#### Example Subscription Input

```json
{
  "topics": ["agentic ai", "llm frameworks"],
  "maxItems": 20,
  "outputFormat": "json",
  "includeCodeExamples": true,
  "subscribe": true,
  "subscriptionTimeUtc": "06:30",
  "subscriptionName": "My Daily AI Updates"
}
```

This will:

- Run immediately with the provided query
- Create/update a schedule named "My Daily AI Updates"
- Run automatically every day at 06:30 UTC
- Deliver fresh insights via your configured output channels

### 🔍 LLM-Powered Internet Search

The Actor can use AI-powered search APIs to find relevant content across the internet. This feature is **optional** and requires API keys.

#### Supported Search Providers

1. **Tavily** (Recommended) - Designed specifically for LLM agents, provides high-quality search results
2. **Perplexity** - AI-powered search with real-time web access
3. **Serper** - Google Search API with fast response times

#### Setup Instructions

##### Step 1: Get Your API Key

Choose one provider and get your API key:

- **Tavily** (Recommended):
  - Sign up at [tavily.com](https://tavily.com)
  - Navigate to your dashboard to get your API key
  - Free tier available

- **Perplexity**:
  - Visit [perplexity.ai/settings/api](https://www.perplexity.ai/settings/api)
  - Generate your API key

- **Serper** (Google Search API):
  - Sign up at [serper.dev](https://serper.dev)
  - Get your API key from the dashboard
  - Free tier: 2,500 searches/month

##### Step 2: Add API Key (Choose One Method)

**Method 1: Direct Input (Easiest - Recommended)**

You can enter API keys directly in the Actor input form when running the Actor:

1. When configuring the Actor run, you'll see secret input fields:
   - **Tavily API Key** - If using Tavily
   - **Perplexity API Key** - If using Perplexity
   - **Serper API Key** - If using Serper
   - **GitHub Token** - Optional, for higher rate limits

2. Simply paste your API key in the appropriate field

3. The field is automatically marked as **Secret** (encrypted and secure)

4. No additional setup needed!

**Method 2: Apify Secrets (For Reusable Keys)**

If you want to reuse the same API keys across multiple Actor runs:

1. Go to **Apify Console** → **Settings** → **Secrets**:
   - Direct link: `https://console.apify.com/account/integrations/secrets`
   - Or: Click your profile icon (top right) → **Settings** → **Integrations** → **Secrets** tab

2. Click **Add Secret** button

3. Add secrets with these names:
   - `TAVILY_API_KEY` - For Tavily
   - `PERPLEXITY_API_KEY` - For Perplexity
   - `SERPER_API_KEY` - For Serper
   - `GITHUB_TOKEN` - Optional, for GitHub

4. The Actor will automatically use these secrets if the input fields are empty

**Priority Order:**

1. Input form secret fields (highest priority)
2. Apify Secrets (fallback)
3. Environment variables (for local development)

##### Step 3: Enable in Actor Input

When running the Actor, set these input parameters:

```json
{
  "topics": ["agentic ai", "llm frameworks"],
  "enableLLMSearch": true,
  "llmSearchProvider": "serper",
  "serperApiKey": "your-serper-api-key-here"
}
```

**Or for Tavily:**

```json
{
  "topics": ["agentic ai", "llm frameworks"],
  "enableLLMSearch": true,
  "llmSearchProvider": "tavily",
  "tavilyApiKey": "your-tavily-api-key-here"
}
```

**Note:**

- Make sure `llmSearchProvider` matches the API key field you fill:
  - `"serper"` → fill `serperApiKey`
  - `"tavily"` → fill `tavilyApiKey`
  - `"perplexity"` → fill `perplexityApiKey`
- If you've set up Apify Secrets, you can leave the API key fields empty and the Actor will use the secrets automatically

#### Example with LLM Search

```json
{
  "topics": ["langchain", "autonomous agents"],
  "maxItems": 15,
  "outputFormat": "json",
  "enableLLMSearch": true,
  "llmSearchProvider": "tavily"
}
```

This will search across all sources including the internet using AI-powered search to find the most relevant recent content.

### 🏗️ Project Structure

```
.
├── .actor/
│   ├── actor.json          # Actor configuration
│   ├── input_schema.json   # Input validation schema
│   └── output_schema.json  # Output schema
├── src/
│   ├── main.ts             # Main orchestrator
│   ├── types.ts            # TypeScript type definitions
│   ├── transformer.ts      # Data transformation logic
│   ├── fetchers/           # Source-specific fetchers
│   │   ├── index.ts
│   │   ├── fetchGitHub.ts
│   │   ├── fetchHuggingFace.ts
│   │   ├── fetchPapersWithCode.ts
│   │   ├── fetchDevTo.ts
│   │   ├── fetchMedium.ts
│   │   └── fetchLLMSearch.ts
│   ├── output/             # Output generators
│   │   └── generators.ts
│   └── utils/              # Utility functions
│       ├── fetch.ts
│       ├── logger.ts
│       └── subscription.ts # Subscription/schedule management
├── package.json
├── tsconfig.json
└── Dockerfile
```

### 🛠️ Development

#### Prerequisites

- Node.js >= 18.0.0
- npm or yarn
- Apify CLI (`npm install -g apify-cli`)

#### Local Development

1. Install dependencies:

```bash
npm install
```

2. Build the project:

```bash
npm run build
```

3. Run locally:

```bash
apify run
```

#### Testing

The Actor can be tested locally with default input or by providing custom input in `storage/key_value_stores/default/INPUT.json`:

```json
{
  "topics": ["agentic ai", "llm frameworks"],
  "maxItems": 10,
  "outputFormat": "json",
  "includeCodeExamples": true,
  "subscribe": false,
  "subscriptionTimeUtc": "06:30",
  "subscriptionName": ""
}
```

#### Testing with Subscription

To test the subscription feature:

```json
{
  "topics": ["agentic ai"],
  "maxItems": 5,
  "outputFormat": "json",
  "includeCodeExamples": true,
  "subscribe": true,
  "subscriptionTimeUtc": "06:30",
  "subscriptionName": "Test Subscription"
}
```

This will create a schedule that runs daily at 06:30 UTC. You can view and manage schedules in the [Apify Console](https://console.apify.com/schedules).

### 📤 Output

#### JSON Format (Default)

- Data is pushed to the default dataset (one item per insight)
- A consolidated JSON file is saved to key-value store as `output.json`

#### Markdown Format

- A formatted Markdown file is saved to key-value store as `output.md`

### 🚢 Deployment

#### Deploy to Apify Platform

1. Log in to Apify:

```bash
apify login
```

2. Deploy the Actor:

```bash
apify push
```

The Actor will be available in your [Apify Console](https://console.apify.com/actors?tab=my).

### 🔧 Technical Details

- **Language**: TypeScript
- **Runtime**: Node.js 18+
- **SDK**: Apify SDK v3
- **Client**: Apify Client v2 (for schedule management)
- **Module System**: ES Modules
- **Build**: TypeScript compiler

### 📝 Code Style

- Fully typed with TypeScript
- Modular architecture with reusable components
- Error handling at every level
- Clean, readable code with comments
- Follows Apify best practices

### 🔄 Future Enhancements

- Additional data sources
- More sophisticated content extraction
- AI-powered summarization
- Custom filtering and ranking
- Webhook notifications
- Multiple subscription frequencies (weekly, monthly)
- Subscription management UI

### 📚 Resources

- [Apify SDK Documentation](https://docs.apify.com/sdk/js)
- [Apify Platform Documentation](https://docs.apify.com/platform)
- [Crawlee Documentation](https://crawlee.dev)
- [Apify Community Discord](https://discord.com/invite/jyEM2PRvMU)

### 📄 License

ISC

# Actor input Schema

## `topics` (type: `array`):

Topics to search for (e.g., 'agentic ai', 'llm frameworks')

## `maxItems` (type: `integer`):

Maximum number of items to collect per source

## `outputFormat` (type: `string`):

Output format: JSON or Markdown

## `includeCodeExamples` (type: `boolean`):

Whether to include code examples in the output

## `subscribe` (type: `boolean`):

Enable daily automatic runs with the same query

## `subscriptionTimeUtc` (type: `string`):

Time of day (UTC) when you want to receive daily updates, e.g. 06:30

## `subscriptionName` (type: `string`):

Optional custom name for this subscription. If not provided, derive a stable name from topics + delivery channel.

## `enableLLMSearch` (type: `boolean`):

Use LLM-powered search APIs (Tavily, Perplexity, or Serper) to find relevant content on the internet. Requires API key in environment variables.

## `llmSearchProvider` (type: `string`):

Choose the search provider: tavily (recommended), perplexity, or serper

## `tavilyApiKey` (type: `string`):

Your Tavily API key for LLM-powered search. Get it at https://tavily.com. Only required if enableLLMSearch is true and llmSearchProvider is 'tavily'.

## `perplexityApiKey` (type: `string`):

Your Perplexity API key for LLM-powered search. Get it at https://www.perplexity.ai/settings/api. Only required if enableLLMSearch is true and llmSearchProvider is 'perplexity'.

## `serperApiKey` (type: `string`):

Your Serper API key for Google Search API. Get it at https://serper.dev. Only required if enableLLMSearch is true and llmSearchProvider is 'serper'.

## `githubToken` (type: `string`):

Optional GitHub personal access token for higher API rate limits (60/hour → 5000/hour). Create at https://github.com/settings/tokens

## Actor input object example

```json
{
  "topics": [
    "agentic ai",
    "llm frameworks"
  ],
  "maxItems": 20,
  "outputFormat": "json",
  "includeCodeExamples": true,
  "subscribe": false,
  "subscriptionTimeUtc": "06:30",
  "subscriptionName": "",
  "enableLLMSearch": false,
  "llmSearchProvider": "tavily"
}
```

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "topics": [
        "agentic ai",
        "llm frameworks"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("transparent_flea/agentic-ai-explorer-developer").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "topics": [
        "agentic ai",
        "llm frameworks",
    ] }

# Run the Actor and wait for it to finish
run = client.actor("transparent_flea/agentic-ai-explorer-developer").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "topics": [
    "agentic ai",
    "llm frameworks"
  ]
}' |
apify call transparent_flea/agentic-ai-explorer-developer --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,transparent_flea/agentic-ai-explorer-developer"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/tlcDaEw7hBo9ZzDnd/builds/AIEKyyP1cq03rKZFF/openapi.json