[dreamwidth/dreamwidth] ac67d5: TheSchwartz deprecation [1 of 4] (#3510)
Feb. 15th, 2026 02:45 pm![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png){kind=small}
github posting in ![[site community profile]](https://www.dreamwidth.org/img/comm_staff.png){kind=small}
changelogBranch: refs/heads/main Home: https://github.com/dreamwidth/dreamwidth Commit: ac67d51ec453222329938ad1508ccadfdca82826 https://github.com/dreamwidth/dreamwidth/commit/ac67d51ec453222329938ad1508ccadfdca82826 Author: Mark Smith mark@dreamwidth.org Date: 2026-02-15 (Sun, 15 Feb 2026)
Changed paths: A .github/workflows/tasks/worker-dw-latest-feed-service.json A .github/workflows/tasks/worker-dw-lazy-cleanup-service.json A .github/workflows/tasks/worker-dw-mass-privacy-service.json A .github/workflows/tasks/worker-dw-synsuck-service.json M .github/workflows/worker-deploy.yml M bin/importadm A bin/worker/dw-latest-feed A bin/worker/dw-lazy-cleanup A bin/worker/dw-mass-privacy A bin/worker/dw-synsuck M bin/worker/schedule-synsuck M cgi-bin/DW/Hooks/SiteSearch.pm M cgi-bin/DW/LatestFeed.pm M cgi-bin/DW/Pay.pm M cgi-bin/DW/Task.pm A cgi-bin/DW/Task/DeleteEntry.pm A cgi-bin/DW/Task/LatestFeed.pm A cgi-bin/DW/Task/MassPrivacy.pm A cgi-bin/DW/Task/SynSuck.pm M cgi-bin/DW/TaskQueue.pm A cgi-bin/DW/TaskQueue/Dedup.pm M cgi-bin/DW/Worker/ContentImporter/LiveJournal/Comments.pm M cgi-bin/DW/Worker/ContentImporter/LiveJournal/Entries.pm M cgi-bin/LJ/Entry.pm M cgi-bin/LJ/MassPrivacy.pm M cgi-bin/LJ/Protocol.pm M cgi-bin/LJ/Talk.pm M config/workers.json A t/taskqueue-dedup.t M terraform/load-balancing.tf
Log Message:
TheSchwartz deprecation [1 of 4] (#3510)
- Add TaskQueue dedup infrastructure and DW::Task enhancements
Add DW::TaskQueue::Dedup module for memcache-based task deduplication. Extend DW::Task with uniqkey/dedup_ttl support for opt-in dedup. Add dedup claim/release logic to DW::TaskQueue dispatch and start_work.
No behavior change: existing TheSchwartz dispatch path is preserved. New infrastructure is available but nothing uses it yet.
Co-Authored-By: Claude Opus 4.5 noreply@anthropic.com
- Add low-risk task classes and switch dispatch paths
Add DW::Task::LatestFeed, DeleteEntry, MassPrivacy, and SynSuck task classes with corresponding dw-* worker scripts. Switch dispatch call sites from TheSchwartz to DW::TaskQueue for these tasks and for SphinxCopier (which already exists as a DW::Task).
Old TheSchwartz worker packages are preserved so existing workers can drain remaining jobs from TheSchwartz queues during transition.
Co-Authored-By: Claude Opus 4.5 noreply@anthropic.com
- Add ECS workflow definitions for low-risk SQS workers
Add dw-latest-feed, dw-lazy-cleanup, dw-mass-privacy, and dw-synsuck worker definitions. Old TheSchwartz workers kept for queue draining.
Co-Authored-By: Claude Opus 4.5 noreply@anthropic.com
- Replace taskopts sentinel with with_dedup() builder and add tests
Replace the fragile taskopts sentinel flag in DW::Task::new with a with_dedup() builder method on the base class. Any task subclass can chain ->with_dedup(uniqkey => ..., dedup_ttl => ...) without needing to override new(), which scales cleanly as more tasks adopt dedup.
Add t/taskqueue-dedup.t with 32 tests covering task construction, with_dedup, Storable round-trip, subclass construction, and DW::TaskQueue::Dedup claim/release/is_pending logic.
Tidyall formatting fixes for files touched by the branch.
Co-Authored-By: Claude Opus 4.6 noreply@anthropic.com
- Fix error handling in new task workers to match TheSchwartz semantics
In TheSchwartz, die was caught and the job was retried. In DW::TaskQueue, die propagates up and crashes the worker process. Wrap all code paths that can die in eval and return FAILED so SQS redelivers the message.
Also fix MassPrivacy missing-args guard to return COMPLETED instead of FAILED — missing args is a permanent failure that retrying won't fix.
Co-Authored-By: Claude Opus 4.6 noreply@anthropic.com
- Add new SQS worker definitions to config/workers.json
Add dw-latest-feed, dw-lazy-cleanup, dw-mass-privacy, and dw-synsuck entries so Terraform creates ECS services for the new SQS-based workers. Scaling matches the old TheSchwartz equivalents they replace.
Co-Authored-By: Claude Opus 4.6 noreply@anthropic.com
- Align terraform load-balancing config with production state
Update listener rules, target group health checks, and listener default actions to match what's actually running in prod, eliminating plan drift. Also convert proxy listeners from target_group_arn shorthand to explicit forward blocks to prevent provider-level format drift, and remove ignore_changes on conditions now that TF matches reality.
Co-Authored-By: Claude Opus 4.6 noreply@anthropic.com
