feat: add news command to fetch AI-curated headlines

Add comprehensive news/trending feature that fetches AI-generated news
headlines from Twitter's "For You" page. This feature specifically
targets the AI-curated news items that Twitter displays under "Today's
News" section, not generic trending hashtags.

- Uses Twitter's ExplorePage GraphQL API endpoint to access the Explore
  page timeline data where AI headlines are located
- AI headlines are identified by the explicit `is_ai_trend: true` flag
  in the API response
- Fallback heuristic detection for full-sentence headlines (5+ words)
  with "News" or time indicators in social context
- Headlines are found in "stories-*" modules within the initialTimeline
  structure under a "Today's News" header
- Follows the existing mixin pattern (withNews) for composability
- Supports both AI-only filtering and mixed trending/AI results

- `bird news` - Fetch news and trending topics (mixed results)
- `bird news --ai-only` - Fetch ONLY AI-curated headlines
- `bird news -n <count>` - Limit number of results
- `bird news --json` - Output as JSON
- `bird news --json-full` - Include raw API response
- `bird news --with-tweets` - Enrich with related tweets
- `bird trending` - Alias for news command
- AI headlines are clearly marked with "AI · " category prefix
- Automatic deduplication of duplicate headlines
- Rich formatting with category, time, post count, and URLs

- src/commands/news.ts - CLI command implementation
- src/lib/twitter-client-news.ts - Core news fetching functionality
- tests/commands.news.test.ts - Command validation tests (7 tests)
- tests/twitter-client-coverage.news.test.ts - API coverage tests (7 tests)

- README.md - Added documentation for news command
- src/cli/program.ts - Registered news command
- src/lib/index.ts - Exported news-related types
- src/lib/twitter-client.ts - Integrated withNews mixin
- src/lib/query-ids.json - Added ExplorePage query ID
- src/lib/twitter-client-constants.ts - Added ExplorePage constant
- src/lib/twitter-client-features.ts - Added buildExploreFeatures()
- scripts/update-query-ids.ts - Added ExplorePage to update script

- All 268 existing tests continue to pass
- Added 14 new tests covering command validation and API functionality
- Tested with real Twitter data confirming AI headline detection
- All TypeScript compilation and linting checks pass

Co-Authored-By: Claude Sonnet 4.5 <[email protected]>
This commit is contained in:
aavetis
2026-01-12 05:32:06 +00:00
committed by Peter Steinberger
co-authored by Claude Sonnet 4.5
parent abc6db411d
commit 57d1498f1d
12 changed files with 1236 additions and 18 deletions
+25 -3
View File
@@ -91,7 +91,12 @@ import { TwitterClient, resolveCredentials } from '@steipete/bird';
const { cookies } = await resolveCredentials({ cookieSource: 'safari' });
const client = new TwitterClient({ cookies });
const result = await client.search('from:steipete', 50);
// Search for tweets
const searchResult = await client.search('from:steipete', 50);
// Fetch news and trending topics
const newsResult = await client.getNews(10, { aiOnly: true, withTweets: true });
```
## Commands
@@ -110,6 +115,8 @@ const result = await client.search('from:steipete', 50);
- `bird bookmarks [-n count] [--folder-id id] [--all] [--max-pages n] [--json]` — list your bookmarked tweets (or a specific bookmark folder); `--max-pages` requires `--all`.
- `bird unbookmark <tweet-id-or-url...>` — remove one or more bookmarks by tweet ID or URL.
- `bird likes [-n count] [--json]` — list your liked tweets.
- `bird news [-n count] [--ai-only] [--with-tweets] [--tweets-per-item n] [--for-you] [--news-only] [--sports] [--entertainment] [--trending-only] [--json]` — fetch news and trending topics from X's Explore tabs.
- `bird trending` — alias for `news` command.
- `bird list-timeline <list-id-or-url> [-n count] [--all] [--max-pages n] [--cursor string] [--json]` — get tweets from a list timeline; `--max-pages` implies `--all`.
- `bird following [--user <userId>] [-n count] [--json]` — list users that you (or another user) follow.
- `bird followers [--user <userId>] [-n count] [--json]` — list users that follow you (or another user).
@@ -212,8 +219,23 @@ When using `--json` with `following`/`followers`, user objects include:
| `followersCount` | number? | Followers count |
| `followingCount` | number? | Following count |
| `isBlueVerified` | boolean? | Blue verified flag |
| `profileImageUrl` | string? | Profile image URL |
| `createdAt` | string? | Account creation timestamp |
| `profileImageUrl` | string? | Profile image URL |
| `createdAt` | string? | Account creation timestamp |
When using `--json` with `news`/`trending`, news objects include:
| Field | Type | Description |
|-------|------|-------------|
| `id` | string | Unique identifier for the news item |
| `headline` | string | News headline or trend title |
| `category` | string? | Category (e.g., "AI · Technology", "Trending", "News") |
| `timeAgo` | string? | Relative time (e.g., "2h ago") |
| `postCount` | number? | Number of posts |
| `description` | string? | Item description |
| `url` | string? | URL to the trend or news article |
| `tweets` | array? | Related tweets (only when `--with-tweets` is used) |
| `_raw` | object? | Raw API response (only when `--json-full` is used) |
## Query IDs (GraphQL)