Monthly Devlog: Local Language Model Scribe & Studio Foundation

Overview

April was a pivotal month focused on advancing the local language model scribe project alongside establishing Watchlight Studio’s foundational infrastructure. The primary technical efforts centered on refining transcription workflows, experimenting with summarization and tagging, and developing a distinctive visual style for related creative assets. Concurrently, I laid groundwork for the studio’s online presence and explored business and monetization strategies to support sustainable development.

Highlights & Progress

Transcription & Summarization Systems

• Implemented a local speech-to-text pipeline using Faster Whisper with a Python script that triggers transcription on speech pauses, outputting time-stamped text directly into Obsidian.
• Developed a custom summarizer integrated within the app, though it currently causes frequent restarts and requires optimization.
• Explored multi-file transcript analysis and scoring methods based on length, quality, and progress to tailor summary sizes; however, initial summaries were verbose and unreliable.
• Auto-tagging was introduced to detect keywords inline, but ambiguity in word meanings limits accuracy.
• Considered backlinking related summaries using KeyBert to enhance navigability and context.
• Mitigated a critical bug causing transcription to halt after the first batch by temporarily disabling notifications; full debugging remains an open challenge.
• Tested alternative models like Mistral Q4 and planned integration of a large language model with an extended token limit (~128,000 tokens) for improved transcript summarization.
• Configured flexible output locations, audio inputs, and file archiving to streamline transcript workflow.
• Polished a tray app interface enabling start/stop toggling and continuous transcription runs.

Studio Infrastructure & Branding

• Officially established Watchlight Studio, securing the domain for five years and setting up email and social media accounts despite bot verification delays.
• Built the studio website using the Hugo static site generator with the Hextra theme, prioritizing independence and flexibility over Quartz.
• Website content includes About Me, studio values, a landing page, and a blog with plans for weekly or monthly devlog posts.
• Initiated infrastructure planning for content tracking and prioritization to support ongoing transparency and engagement.

Visual & Asset Development

• Experimented with Blender geometry and render nodes to create a unique low poly and pixel art style.
• Implemented Bezier pathing through a terrain model with tile materials assigned based on distance along the path.
• Encountered challenges classifying tiles into categories such as center, edge, corner, inside convex, and outside concave.
• Dirt tile textures require further refinement to better align with the pixel art aesthetic.
• Tested auto-tiling methods for generating adaptable low poly assets across different terrain meshes; this remains experimental.

Business & Monetization Exploration

• Investigated business structures, weighing sole proprietorship versus LLC, considering income requirements, compliance, and risk factors like lawsuits and maintenance costs.
• Brainstormed monetization strategies including devlogs, YouTube content, Patreon, subscription sites with ads, asset sales, and selling the transcription app.
• Pricing and strategic prioritization remain undecided, with an ongoing debate whether to focus on completing the scribe for commercial release or pivot toward building a project manager AI agent.

Reflections

• The integration of Faster Whisper with Obsidian for real-time transcription was a significant technical success, providing a solid foundation for the scribe project.
• The custom summarizer shows promise but highlighted the need for stability improvements and more sophisticated summarization strategies, especially for multi-file transcripts.
• Auto-tagging’s struggles with ambiguous terms revealed the complexity of natural language understanding in local models and pointed toward the need for enhanced disambiguation approaches.
• Establishing the studio’s online presence and branding early has created a motivating framework for ongoing work, though initial setup was slowed by external factors like bot checks.
• Visual asset experimentation underscored the difficulty of blending low poly and pixel art styles, particularly in tile classification and texture consistency.
• Business considerations introduced a new dimension of complexity, reminding me that technical progress must be balanced with sustainable operational planning.
• Temporarily disabling notifications to fix the transcription halt was a pragmatic workaround, but the underlying issue remains a priority to resolve for robustness.

Next Steps

• Fully debug and resolve the notification-related transcription halt to enable uninterrupted long-form recording.
• Optimize the custom summarizer for stability and experiment with refined summarization interfaces to improve multi-file transcript quality.
• Enhance auto-tagging accuracy by exploring context-aware disambiguation methods or local model assistance.
• Develop backlinking strategies leveraging KeyBert or similar tools to create meaningful connections between summaries.
• Continue refining tile classification logic and dirt tile textures to solidify the pixel art style and improve visual coherence.
• Decide on project prioritization between completing the scribe for commercial release versus initiating the project manager AI agent.
• Finalize business structure and monetize strategy to align development efforts with sustainable funding.

Keynotes

Decisions & Observations

• Decided to implement a local speech-to-text pipeline using Faster Whisper integrated with Obsidian for real-time transcription.

Issues & Next Steps

• Needs to resolve a critical bug causing transcription to halt after the first batch, currently mitigated by disabling notifications.
• Will optimize the custom summarizer to improve stability and multi-file transcript summarization quality.
• Will explore context-aware disambiguation methods to enhance auto-tagging accuracy amid ambiguous terms.
• Will develop backlinking strategies using KeyBert or similar tools to improve summary navigation and context.
• Needs to refine tile classification logic and dirt tile textures to achieve a consistent low poly and pixel art visual style.
• Will decide on project prioritization between completing the scribe for commercial release or pivoting to build a project manager AI agent.
• Needs to finalize the business structure and monetization strategy to support sustainable development and align with project goals.

Generated with Nucleate by Watchlight Studio — distilled from full transcription.