Files

T

xiaoming 069af30dba Initial import: Music_Server, MusicFree, catalog-sync

2026-05-23 16:51:14 +08:00

18 KiB

Raw Blame History

Playlist File Run Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Add a --playlist-file option to musicdl.catalogsync so run can process only playlists listed in a text file while preserving the current default workflow when the option is absent.

Architecture: Keep the existing collect -> sync -> download path unchanged and add a narrow file-driven branch in run. Parse playlist-file lines into normalized playlist import entries, upsert them into the existing catalog tables under a manual_file pool, then sync and download only the referenced playlist IDs.

Tech Stack: Python stdlib (pathlib, dataclasses, urllib.parse), click, existing musicdl.catalogsync modules, unittest

Task 1: Add file playlist parsing and manual-file pool support

Files:

Create: musicdl/catalogsync/manual_playlists.py
Modify: musicdl/catalogsync/models.py
Modify: musicdl/catalogsync/repository.py
Test: tests/catalogsync/test_manual_playlists.py
Step 1: Write failing tests for playlist-file parsing

import tempfile
import unittest
from pathlib import Path


class ManualPlaylistParsingTests(unittest.TestCase):
    def test_parse_playlist_file_supports_url_and_platform_url_lines(self):
        from musicdl.catalogsync.manual_playlists import parse_playlist_file

        with tempfile.TemporaryDirectory(ignore_cleanup_errors=True) as tmpdir:
            playlist_file = Path(tmpdir) / "playlists.txt"
            playlist_file.write_text(
                "\n".join(
                    [
                        "# comment",
                        "https://music.163.com/#/playlist?id=17745989905",
                        "qq,https://y.qq.com/n/ryqq/playlist/7707261125",
                        "https://music.163.com/#/playlist?id=17745989905",
                        "",
                    ]
                ),
                encoding="utf-8",
            )

            parsed = parse_playlist_file(playlist_file)

        self.assertEqual(5, parsed.total_lines)
        self.assertEqual(0, parsed.skipped_lines)
        self.assertEqual(2, len(parsed.entries))
        self.assertEqual("netease", parsed.entries[0].platform)
        self.assertEqual("17745989905", parsed.entries[0].remote_id)
        self.assertEqual("qq", parsed.entries[1].platform)
        self.assertEqual("7707261125", parsed.entries[1].remote_id)

Step 2: Run the focused parser test to verify it fails

Run: python -m unittest tests.catalogsync.test_manual_playlists.ManualPlaylistParsingTests.test_parse_playlist_file_supports_url_and_platform_url_lines -v Expected: FAIL with import error for musicdl.catalogsync.manual_playlists or missing parser helpers

Step 3: Implement the parser module and manual-file pool helper

from __future__ import annotations

from dataclasses import dataclass
from pathlib import Path

from .models import PlaylistCandidate


@dataclass(slots=True)
class ParsedPlaylistFile:
    entries: list[PlaylistCandidate]
    total_lines: int
    skipped_lines: int


def parse_playlist_file(path: str | Path) -> ParsedPlaylistFile:
    raise NotImplementedError

    def get_or_create_manual_file_pool(self, playlist_file: str | Path) -> int:
        resolved = Path(playlist_file).resolve()
        return self.upsert_playlist_pool(
            platform="manual",
            pool_kind="manual_file",
            external_id=f"manual_file:{resolved}",
            name=f"Manual File Import: {resolved.name}",
            url=str(resolved),
        )

Step 4: Re-run the focused parser tests

Run: python -m unittest tests.catalogsync.test_manual_playlists -v Expected: PASS

Step 5: Commit

git add musicdl/catalogsync/manual_playlists.py musicdl/catalogsync/models.py musicdl/catalogsync/repository.py tests/catalogsync/test_manual_playlists.py
git commit -m "feat: add playlist file parsing support"

Task 2: Add manual playlist import and targeted sync

Files:

Modify: musicdl/catalogsync/services.py
Modify: musicdl/catalogsync/repository.py
Test: tests/catalogsync/test_manual_playlists.py
Test: tests/catalogsync/test_services.py
Step 1: Write failing tests for manual playlist import and targeted sync

    def test_import_manual_playlists_creates_manual_pool_and_returns_playlist_ids(self):
        from musicdl.catalogsync.db import initialize_database
        from musicdl.catalogsync.manual_playlists import parse_playlist_file
        from musicdl.catalogsync.repository import CatalogRepository
        from musicdl.catalogsync.services import CatalogSyncService

        with tempfile.TemporaryDirectory(ignore_cleanup_errors=True) as tmpdir:
            db_path = Path(tmpdir) / "catalogsync.db"
            playlist_file = Path(tmpdir) / "playlists.txt"
            playlist_file.write_text(
                "https://music.163.com/#/playlist?id=17745989905\n",
                encoding="utf-8",
            )
            initialize_database(db_path).close()
            repo = CatalogRepository(db_path)
            service = CatalogSyncService(repo)

            parsed = parse_playlist_file(playlist_file)
            playlist_ids = service.import_manual_playlists(playlist_file, parsed.entries)

        self.assertEqual(1, len(playlist_ids))
        self.assertEqual(1, repo.count_rows("playlists"))
        self.assertEqual(
            1,
            len(repo.list_pool_playlist_links(repo.get_or_create_manual_file_pool(playlist_file))),
        )

    def test_sync_specific_playlists_only_processes_requested_playlist_ids(self):
        from unittest.mock import patch

        from musicdl.catalogsync.db import initialize_database
        from musicdl.catalogsync.models import PlaylistCandidate
        from musicdl.catalogsync.repository import CatalogRepository
        from musicdl.catalogsync.services import CatalogSyncService

        with tempfile.TemporaryDirectory(ignore_cleanup_errors=True) as tmpdir:
            db_path = Path(tmpdir) / "catalogsync.db"
            initialize_database(db_path).close()
            repo = CatalogRepository(db_path)
            service = CatalogSyncService(repo)

            playlist_a = repo.upsert_playlist(
                PlaylistCandidate(
                    platform="netease",
                    pool_kind="manual_file",
                    remote_id="17745989905",
                    name="Playlist A",
                    url="https://music.163.com/#/playlist?id=17745989905",
                )
            )
            playlist_b = repo.upsert_playlist(
                PlaylistCandidate(
                    platform="netease",
                    pool_kind="manual_file",
                    remote_id="17729789137",
                    name="Playlist B",
                    url="https://music.163.com/#/playlist?id=17729789137",
                )
            )

            with patch.object(service, "resolve_playlist_song_infos", return_value=[] ) as resolver:
                service.sync_specific_playlists([playlist_b])

        called_row = resolver.call_args[0][0]
        self.assertEqual(playlist_b, int(called_row["id"]))

Step 2: Run the focused services tests to verify they fail

Run: python -m unittest tests.catalogsync.test_manual_playlists tests.catalogsync.test_services -v Expected: FAIL with missing import_manual_playlists, list_playlists_by_ids, and sync_specific_playlists

Step 3: Implement manual playlist import and targeted sync entry points

    def list_playlists_by_ids(self, playlist_ids: list[int]) -> list[sqlite3.Row]:
        placeholders = ", ".join("?" for _ in playlist_ids)
        return self._fetchall(
            f"SELECT * FROM playlists WHERE id IN ({placeholders}) ORDER BY id ASC",
            tuple(playlist_ids),
        )

    def import_manual_playlists(self, playlist_file: str | Path, candidates: list[PlaylistCandidate]) -> list[int]:
        pool_id = self.repository.get_or_create_manual_file_pool(playlist_file)
        playlist_ids = []
        for candidate in candidates:
            playlist_id = self.repository.upsert_playlist(candidate)
            self.repository.link_pool_playlist(pool_id, playlist_id)
            playlist_ids.append(playlist_id)
        return playlist_ids

    def sync_specific_playlists(self, playlist_ids: list[int]) -> int:
        processed = 0
        for playlist_row in self.repository.list_playlists_by_ids(playlist_ids):
            song_infos = self.resolve_playlist_song_infos(playlist_row)
            for pool_id in self.repository.get_pool_ids_for_playlist(int(playlist_row["id"])):
                self.store_playlist_songs(int(playlist_row["id"]), pool_id, song_infos)
            processed += len(song_infos)
        return processed

Step 4: Re-run the focused services tests

Run: python -m unittest tests.catalogsync.test_manual_playlists tests.catalogsync.test_services -v Expected: PASS

Step 5: Commit

git add musicdl/catalogsync/services.py musicdl/catalogsync/repository.py tests/catalogsync/test_manual_playlists.py tests/catalogsync/test_services.py
git commit -m "feat: add manual playlist import and targeted sync"

Task 3: Add targeted download planning for selected playlists

Files:

Modify: musicdl/catalogsync/downloader.py
Modify: musicdl/catalogsync/repository.py
Test: tests/catalogsync/test_services.py
Step 1: Write a failing test for limiting downloads to selected playlist IDs

    def test_download_planner_can_limit_queue_to_specific_playlists(self):
        from musicdl.catalogsync.db import initialize_database
        from musicdl.catalogsync.models import CatalogSong, PlaylistCandidate
        from musicdl.catalogsync.repository import CatalogRepository
        from musicdl.catalogsync.downloader import DownloadPlanner

        with tempfile.TemporaryDirectory(ignore_cleanup_errors=True) as tmpdir:
            db_path = Path(tmpdir) / "catalogsync.db"
            initialize_database(db_path).close()
            repo = CatalogRepository(db_path)

            playlist_a = repo.upsert_playlist(
                PlaylistCandidate(
                    platform="qq",
                    pool_kind="manual_file",
                    remote_id="7707261125",
                    name="Playlist A",
                    url="https://y.qq.com/n/ryqq/playlist/7707261125",
                )
            )
            playlist_b = repo.upsert_playlist(
                PlaylistCandidate(
                    platform="qq",
                    pool_kind="manual_file",
                    remote_id="7578943835",
                    name="Playlist B",
                    url="https://y.qq.com/n/ryqq/playlist/7578943835",
                )
            )
            song_a = repo.upsert_song(CatalogSong(platform="qq", remote_song_id="song-a", name="A"))
            song_b = repo.upsert_song(CatalogSong(platform="qq", remote_song_id="song-b", name="B"))
            repo.link_playlist_song(playlist_a, song_a, 1)
            repo.link_playlist_song(playlist_b, song_b, 1)

            planner = DownloadPlanner(repo)
            queue = planner.build_download_queue(playlist_ids=[playlist_a])

        self.assertEqual([song_a], [item["song_id"] for item in queue])

Step 2: Run the focused download planner test to verify it fails

Run: python -m unittest tests.catalogsync.test_services.CatalogServiceTests.test_download_planner_can_limit_queue_to_specific_playlists -v Expected: FAIL because playlist_ids is unsupported

Step 3: Implement repository and downloader filtering by playlist IDs

    def list_pending_download_songs(
        self,
        sources: list[str] | None = None,
        limit: int | None = None,
        playlist_ids: list[int] | None = None,
    ) -> list[sqlite3.Row]:
        query = """
            SELECT DISTINCT s.*
            FROM songs s
            JOIN playlist_songs ps ON ps.song_id = s.id
            WHERE NOT EXISTS (
                SELECT 1
                FROM file_locations fl
                JOIN file_assets fa ON fa.id = fl.file_asset_id
                JOIN storage_backends sb ON sb.id = fl.backend_id
                WHERE fa.song_id = s.id
                  AND fl.status = 'active'
                  AND sb.backend_type = 'local_fs'
            )
        """

    def build_download_queue(
        self,
        sources: list[str] | None = None,
        limit: int | None = None,
        playlist_ids: list[int] | None = None,
    ) -> list[dict]:
        rows = self.repository.list_pending_download_songs(
            sources=sources,
            limit=limit,
            playlist_ids=playlist_ids,
        )
        queue = []
        for row in rows:
            if self.repository.song_has_active_local_file(int(row["id"])):
                continue
            item = dict(row)
            item["song_id"] = int(row["id"])
            queue.append(item)
        return queue

Step 4: Re-run the focused download tests

Run: python -m unittest tests.catalogsync.test_services -v Expected: PASS

Step 5: Commit

git add musicdl/catalogsync/downloader.py musicdl/catalogsync/repository.py tests/catalogsync/test_services.py
git commit -m "feat: limit downloads to selected playlists"

Task 4: Wire `--playlist-file` into the CLI

Files:

Modify: musicdl/catalogsync/cli.py
Modify: tests/catalogsync/test_cli.py
Test: tests/catalogsync/test_manual_playlists.py
Step 1: Write failing CLI tests for the file-driven run path

    def test_run_command_uses_playlist_file_branch_without_collect(self):
        from musicdl.catalogsync.cli import cli

        runner = CliRunner()
        with tempfile.TemporaryDirectory(ignore_cleanup_errors=True) as tmpdir:
            db_path = Path(tmpdir) / "catalogsync.db"
            playlist_file = Path(tmpdir) / "playlists.txt"
            playlist_file.write_text(
                "https://music.163.com/#/playlist?id=17745989905\n",
                encoding="utf-8",
            )
            with patch("musicdl.catalogsync.cli.CatalogSyncApplication") as app_cls:
                app = app_cls.return_value

                result = runner.invoke(
                    cli,
                    [
                        "run",
                        "--db",
                        str(db_path),
                        "--library-root",
                        str(Path(tmpdir) / "library"),
                        "--playlist-file",
                        str(playlist_file),
                    ],
                )

        self.assertEqual(0, result.exit_code, msg=result.output)
        app.collect_playlists.assert_not_called()
        app.run_playlist_file.assert_called_once()

Step 2: Run the focused CLI test to verify it fails

Run: python -m unittest tests.catalogsync.test_cli.CatalogCliTests.test_run_command_uses_playlist_file_branch_without_collect -v Expected: FAIL because --playlist-file and run_playlist_file do not exist

Step 3: Implement the CLI branch and application method

    def run_playlist_file(self, playlist_file: str, sources: list[str] | None = None, limit: int | None = None):
        parsed = parse_playlist_file(playlist_file)
        playlist_ids = self.service.import_manual_playlists(playlist_file, parsed.entries)
        if limit is not None:
            playlist_ids = playlist_ids[:limit]
        synced = self.service.sync_specific_playlists(playlist_ids)
        downloaded = self.downloader.download_pending(
            self.library_root,
            sources=sources,
            limit=limit,
            playlist_ids=playlist_ids,
        )
        return parsed, synced, downloaded

Step 4: Re-run the focused CLI tests

Run: python -m unittest tests.catalogsync.test_cli -v Expected: PASS

Step 5: Commit

git add musicdl/catalogsync/cli.py tests/catalogsync/test_cli.py tests/catalogsync/test_manual_playlists.py
git commit -m "feat: add playlist file run option"

Task 5: Update docs and run final verification

Files:

Modify: docs/catalogsync.md
Modify: README.md
Test: tests/catalogsync/test_manual_playlists.py
Test: tests/catalogsync/test_cli.py
Test: tests/catalogsync/test_services.py
Step 1: Update docs with --playlist-file examples and file format rules

### 从文件读取歌单

```bash
musicdl-catalogsync run --db D:\catalogsync\catalogsync.db --library-root E:\MusicLibrary --playlist-file D:\lists\playlists.txt

文件支持：

一行一个歌单 URL
平台,歌单URL
# 注释行


- [ ] **Step 2: Run the full catalogsync test suite**

Run: `python -m unittest discover -s tests/catalogsync -v`
Expected: PASS

- [ ] **Step 3: Run a manual smoke flow using a temporary playlist file**

Run:
```bash
python -m musicdl.catalogsync.cli init-db --db .tmp\playlist-file\catalogsync.db --library-root .tmp\playlist-file\library
python -m musicdl.catalogsync.cli run --db .tmp\playlist-file\catalogsync.db --library-root .tmp\playlist-file\library --playlist-file .tmp\playlist-file\playlists.txt

Expected:

command exits 0
the file-driven branch skips collect
playlists from the file are imported and processed
Step 4: Commit

git add docs/catalogsync.md README.md
git commit -m "docs: add playlist file run usage"

18 KiB Raw Blame History

Playlist File Run Implementation Plan

Task 1: Add file playlist parsing and manual-file pool support

Task 2: Add manual playlist import and targeted sync

Task 3: Add targeted download planning for selected playlists

Task 4: Wire --playlist-file into the CLI

Task 5: Update docs and run final verification

18 KiB

Raw Blame History

Task 4: Wire `--playlist-file` into the CLI