What's the intended use case for this? It seems like you'd create slop videos for social media. I'd love to see more AI use cases that aren't: uninteresting content people would prefer to avoid.
It’s actually designed for your own gameplay—it scans hours long raw session to find the best highlights and clips them into shorts. It's more about automating the tedious editing process for your own content rather than generating "slop" from scratch.
Can I use this for other use cases instead of game videos?
I want to create film-style scenes, cinematic elements, and smooth motion effects.
I’m also thinking of deploying it as a SaaS and using it for video creation features in my app: https://picxstudio.com/
Definitely. The architecture is modular—just swap the LLM prompts for 'cinematic' styles. It's headless and dockerized, so it fits well as a SaaS backend worker
I built this because I was tired of "AI tools" that were just wrappers around expensive APIs with high latency. As a developer who lives in the terminal (Arch/Nushell), I wanted something that felt like a CLI tool and respected my hardware.
The Tech:
GPU Heavy: It uses decord and PyTorch for scene analysis. I’m calculating action density and spectral flux locally to find hooks before hitting an LLM.
Local Audio: I’m using ChatterBox locally for TTS to avoid recurring costs and privacy leaks.
Rendering: Final assembly is offloaded to NVENC.
Looking for Collaborators: I’m currently looking for PRs specifically around:
Intelligent Auto-Zoom: Using YOLO/RT-DETR to follow the action in a 9:16 crop.
Voice Engine Upgrades: Moving toward ChatterBoxTurbo or NVIDIA's latest TTS.
It's fully dockerized, and also has a makefile. Would love some feedback on the pipeline architecture!
I don't get this reasoning. You were tired of LLM wrappers, but what is your tool? These two requirements (felt like a CLI and respects your hardware) do not line up.
Still a cool tool though! Although it seems partly AI generated.
Seems like the post you're replying to has since been edited to clarify that he's referring to the wrappers that rely on third party AI APIs over the internet rather than running locally.
Fair points all around. To be transparent: yes, I used an AI coding assistant (Antigravity) to help with the heavy lifting of refactoring the original legacy code and drafting the README. I’m with @rustyhancock on this—I’d rather focus my brainpower on the pipeline logic and hardware integration than on writing boilerplate and Markdown.
However, orchestrating things like decord with CUDA kernels, managing VRAM across parallel processes, and getting audio sync right with local TTS requires a deep understanding of the stack. An LLM can help write a function, but it won't solve the architectural 'glue' needed to make it a reliable CLI tool.
The project is open-source precisely because it’s a work in progress. It needs the 'human touch' for things like the RT-DETR auto-zoom and more nuanced video editing logic. PRs are more than welcome—I'd love to see where the community can push this beyond its current state.
Fair point. I used SOTA models for the analysis to prioritize quality, but since the heavy media processing is local, API costs stay negligible (or free).
The architecture is modular, though—you can definitely swap in a local LLM for a fully air-gapped setup.
big fan of the 'respects my hardware' philosophy. i feel like 90% of ai tools right now are just expensive middleware for openai, so seeing something that actually leverages local compute (and doesn't leak data) is refreshing
What's the intended use case for this? It seems like you'd create slop videos for social media. I'd love to see more AI use cases that aren't: uninteresting content people would prefer to avoid.
It’s actually designed for your own gameplay—it scans hours long raw session to find the best highlights and clips them into shorts. It's more about automating the tedious editing process for your own content rather than generating "slop" from scratch.
Can I use this for other use cases instead of game videos? I want to create film-style scenes, cinematic elements, and smooth motion effects. I’m also thinking of deploying it as a SaaS and using it for video creation features in my app: https://picxstudio.com/
Definitely. The architecture is modular—just swap the LLM prompts for 'cinematic' styles. It's headless and dockerized, so it fits well as a SaaS backend worker
I built this because I was tired of "AI tools" that were just wrappers around expensive APIs with high latency. As a developer who lives in the terminal (Arch/Nushell), I wanted something that felt like a CLI tool and respected my hardware.
The Tech:
Looking for Collaborators: I’m currently looking for PRs specifically around: It's fully dockerized, and also has a makefile. Would love some feedback on the pipeline architecture!I don't get this reasoning. You were tired of LLM wrappers, but what is your tool? These two requirements (felt like a CLI and respects your hardware) do not line up.
Still a cool tool though! Although it seems partly AI generated.
Seems like the post you're replying to has since been edited to clarify that he's referring to the wrappers that rely on third party AI APIs over the internet rather than running locally.
I've started including a statement of AI usage in my docs.
HN is a niche audience but it seems like it's the first question everyone has when opening a repo.
Which is odd because the first question we should have is, does it work.
Personally I can't see myself ever writing the bulk of the README again, life's too short.
Fair points all around. To be transparent: yes, I used an AI coding assistant (Antigravity) to help with the heavy lifting of refactoring the original legacy code and drafting the README. I’m with @rustyhancock on this—I’d rather focus my brainpower on the pipeline logic and hardware integration than on writing boilerplate and Markdown.
However, orchestrating things like decord with CUDA kernels, managing VRAM across parallel processes, and getting audio sync right with local TTS requires a deep understanding of the stack. An LLM can help write a function, but it won't solve the architectural 'glue' needed to make it a reliable CLI tool.
The project is open-source precisely because it’s a work in progress. It needs the 'human touch' for things like the RT-DETR auto-zoom and more nuanced video editing logic. PRs are more than welcome—I'd love to see where the community can push this beyond its current state.
I think my life's too short to ever read your READMEs.
> Multi-Provider Support: Choose between OpenAI (GPT-5-mini, GPT-4o) or Google Gemini for scene analysis
This is the first sentence in your features section, so it is not strange if users don't understand if this tool is running locally or not.
Fair point. I used SOTA models for the analysis to prioritize quality, but since the heavy media processing is local, API costs stay negligible (or free). The architecture is modular, though—you can definitely swap in a local LLM for a fully air-gapped setup.
It looks like it’s written by a LLM
Guilty as charged. I used Antigravity to handle the refactoring and docs so I could stay focused on the CUDA and VRAM orchestration.
How much memory do you need locally? Is a rtx 3090 with 24gb enough?
Yes, more than enough. I have rtx4080 laptop gpu with 12gb vram.
Wow, great job.
I did smth similar 4 years ago with YOLO ultralytics.
Back then I used chat messsges spike as one of several variables to detect highs and fails moments. It needed a lot a human validation but was so fun.
Keep going
Great idea. Integrating YOLO for 'Action Following' is high on the roadmap—I'd love a PR for that if you're interested!
big fan of the 'respects my hardware' philosophy. i feel like 90% of ai tools right now are just expensive middleware for openai, so seeing something that actually leverages local compute (and doesn't leak data) is refreshing