You should sell this to Lawyers and other professionals who bill per hour to reconstruct their billables for the day without missing anything. They would pay big money for something that recovered forgotten(unbilled) work throughout the day.
One of my first ever bug reports, was a submission to a company that made legal software.
In particular, it was a document management system built as a plugin for MS Outlook. (ew)
Most users, had no issue. However for one user, a lawyer, in particular, she would open and close a bunch of documents (using the built in pdf viewer) and then the application would crash, taking outlook with it, often requiring a restart.
I went over to view the behavior, and she was some kind of robot. Unlike her peers, she had 12 documents open at once, and she could update and bill (in minimum 6 or 7 minute increments) 12 customers cases in 15 minutes. It was like meeting the Usain Bolt of law practitioners. My back of the napkin math is that she billed like 3-4 hours for every hour she was online.
Open Email
Load Attachment
Review Attachment
Reply to email
Assign Email thread to case number
Close attachment.
12 times in 15 minutes.
The bug was that, after loading ~6 pdfs, the application would back off and wait to deallocate the memory. It would then later, randomly decide to write to that memory when another pdf was loaded, and go kaput.
Just to replicate the issue, I had to close and reopen pdfs so quickly my hands hurt.
It took 3 revisions of the bug report to get the software company to accept it and resolve it. And even then I think the pdf limit just increased, before we submitted another report and had it resolved permanently.
On that note, the principal of another law firm I supported would require us to cleanse his personal laptop of porn themed golf games he had downloaded on a regular basis.
The impression I get is that, lawyers work but the work is just unevenly distributed.
Digging deep in to my memory, I recall that the user had at least one instance of Strip Putt Putt. I cant conceive of how that answers your question but its the best I can do.
I get the sentiment behind your comment but I have a few lawyers in the family and they work round the clock. They might be in meetings or pouring over documents all day that might not look like work to the average software engineer but trust me, they do work hard. And it's true for everyone - from junior interns to senior partners.
> They might be in meetings or pouring over documents all day
FWIW it’s “poring over” when reading carefully.
From Merriam-Webster
“As a verb, pore means "to gaze intently" or "to reflect or meditate steadily." The verb pour has meanings referring to the falling or streaming of liquid (or things that move like liquid).”
Wouldn’t you? If I switch context and interrupt my flow to answer a question I’m losing at least 20 mins to regaining focus, why shouldn’t that be reflected in billing?
Knowledge work is knowledge work, no point belittling colleagues in a different profession.
Thats how MSPs operate too. At least the good ones. Billing increments are sometimes as low as 6 minutes, or as high as 30. 15 minutes is average in my experience.
I'm a litigation legal admin - I have been for 25-30 years. I instantly brought this up to an associate, telling them, "Maybe not now, but before you retire, this'll be the norm in the industry."
She had been complaining the day before about having to reconstruct a huge bunch of little 0.1 entries involving e-mails to various individuals in cases. If it could be done automatically, through a local LLM? chef's kiss
Trust me, law is definitely where you want to land this thing.
In all honesty, I have absolutely no negotiating power or decision-making authority for my firm, but it's a big one -- if that's a direction you want to go, can't guaranty I can swing enough weight, but I probably could find you the right people to talk to, give you an introduction.
I'll also have to add, though, that you'd have to figure out a way for it to be cross-platform or live outside just macOS. Unfortunately, that's a very uncommon choice in the legal world (or anywhere else).
Yeah, but it's written in a modular way and extending it is not as painful as one would expect. I actually went that way and wrote a couple of custom watchers for things like that.
Every lawyer in the USA that I've ever worked with also bills in 6 minute increments. Which means every email is 6 minutes. Every phone call is at least 6 minutes. etc.
Isn't this better though? The alternative being that every email or phone call is an hour, or that they batch it up based on gut feel. If I get a phone call that could easily steal 6 minutes of focus time (note that I specify focus time, even if the call is only 30 seconds you'd have to mentally switch tasks back and forth).
I’m not sure that 6 minutes is a useful denomination of focus time. Maybe legal work is too different from IT and it makes sense there, but when I need to focus on something, 15 minutes is the smallest amount I would allocate for a task.
Really nice! I currently use ActivityWatch for tracking tasks on PC.
Some things I would like to be able to do with software like this:
- Identify the 'spark' of a distraction. For example, opening my email inbox to read a specific email also shows me many unrelated emails. These can easily be the cause of a 5-15 minute distraction. This information is often actionable. I installed browser plugins to hide my youtube suggested videos and my distractions went down. I made sure to close all unused windows to avoid catching a glimpse of unrelated work.
- Identify repeated tasks, and the cadence of those tasks. Do I manually make an invoice once a week for a particular edge case? Is the process basically identical every time. Could this be automated?
- How was I feeling before, during and after a task. (This is a very broad and intentionally not well-defined question, but I think it has the most promise for improving procrastination and task initiation).
Yep, helping people understand their distraction patterns would be an amazing feature. I find myself doing the same thing, funnily enough I also have that same Youtube extension.
I love the product concept but the fact this person has an almost empty github and suddenly launches an app that can easily be spyware concerns me a lot :). A lot of security concerns with password etc.
I would not be comfortable sending my bank info passwords and all sorts of other sensitive data that I input and see on my screen to Gemini. How much is the qualitative performance difference with a local model?
If I had to put a grade on my own experience and evals, Gemini 2.5 pro produces A- results and qwen2.5vl is maybe like B-/C+. Obviously everything's nondetermistic, so it's hard to guarantee a level of quality.
I'm reading through papers that suggest it should be possible to get SOTA performance on local models via distillation, and that's what I'll experiment with next.
Also, if your not using an enterprise edition of gemini where your data is not used for model training, your sensitive data prompts and responses is 100% available to google.
Google owns my email, browser, phone operating system, and a small amount of passwords. I assume that it has already stolen all my confidential data by now.
I haven't seen this mentioned, but I immediately thought this could be a great tool for folks with ADHD. The potential for seeing what kinds of things regularly trigger distraction (I know, everything, right?) and any patterns that exist (i.e. every time I make a git commit, I go check Hacker News and lose 15 minutes). As well as being able to review day that was captured automatically is huge. The best success I had with tracking what I did was when I used to use TimeRescue to ensure I had accurate record of hours for clients, but every attempt to use anything that requires manual entry fails very quickly (either too distracting everytime I use it, or I literally just forget to use it).
Going a step further, "real time" (given processing delay) to help stay on task when the focus has shifted to something unrelated (maybe allow the individual to define this or say yes/no to train the prompts as it goes).
Anyways, it looks great. I also liked the _idea_ of Windows Recall, so to see something like this that can be privacy first is really nice.
Pretty nice. How does it handle multiple displays? I've set it up with local Ollama, and it seems to only record and analyze one of my two secondary displays. It would be ideal if I can select which one is used if the recording is limited to a single display, or even better if it can record and analyze the entire multi-monitor desktop surface.
edit Nvm, it seems it always records the display that is currently in focus. That is probably the better way to handle it, since it automatically solves the "ignore what's shown but not interacted with on secondary displays" problem.
This was my question also, I think "even better if it can record and analyze the entire multi-monitor desktop surface" would be the best option. I don't know what the impact of that would be on both recording size and AI processing time, but just because one monitor is focused doesn't always mean what's happening on another is ignored. Some examples: an ongoing meeting or watching a video on one screen while taking notes on another; or coding on one screen and a browser/app auto-refreshing on another.
Yep, you figured out how it works! That was the easiest solution I could come up with. I'm sure theres additional context on other screens but this was a good 90/10 solution.
Thanks! Between my friends and I, it's about a 50/50 split between local and cloud. I think it's great to be able to pick the tradeoff between quality/privacy based on your own privacy preferences.
Great project! I’ve had a similar experience with Rewind and the related privacy concerns. A quick thought: if I recall correctly, Rewind performs OCR locally, so it only needs to send textual data. Since you’re focusing on macOS, you could rely on VNRecognizeTextRequest and skip the extra OCR complexity. It might also help to detect and mask sensitive information with lightweight models (e.g., BERT), especially when leveraging cloud-based AI.
On one hand I'm super enthusiastic about your project.
This could help battle procrastination, organize your time in a long run, bill your clients more efficiently, etc. 20 years younger, hyper productive me would kill for such product.
But then I recall when I accidently suggested TimeRescue to my boss at one time, and suddenly he was skimming though everyones daily logs to see if they're spending 100% of their times in business facing apps.
When I first heard about "covid mouse mover devices" that faked activity for remote workers I thought it was a joke. Seriously.
But I'm afraid this is the dystopian future. Employers constantly looking at your screen and getting spreadsheets with your daily effort.
This was my first thought too. The last generation of activity tracking, while still dystopian, was a little different at least in that it was mainly statistical. So action-wise, it might point managers at "potential problems," but doesn't make its way into a performance review (e.g. "your mouse only traveled 81.72 screen-miles this quarter, 2 standard deviations below the mean, while you also scored the lowest on number of keystrokes with vscode as the active window..."). If a manager really wanted to summarize exactly what was done they had to spend an almost equal amount of time to watch. To some degree, this alleviates that.
Yea, honestly I would hate if people used this to track _other_ people, especially bosses. I wanted to build something that gave people more agency to do more with their precious time, but there definitely is a fine line here.
Probably someone long ago said the same about hope for pointy sticks not being used just for hunting animals. Yet someone will likely make a pointy stick whether you do it or not.
Woah this is fab; much less cognitive load than manually using a time tracker. And I'm glad that there's a local option and a "BYO key" option for privacy!
Feel like something of this shape should have existed for a while, but this is very well executed!
if you are on a Zoom/video call, does anyone know if you would have to declare that your "recording" it? I'm thinking more from the legal perspective of wiretapping/consent laws. If you have live transcripts/subtitles does that change any legal requirement.
Yes, in my state it's generally illegal to take a screenshot of a zoom call without announcing you are recording the conversation. I'm not certain, but I think the issue is the storage of the 1fps video, not the AI summary.
I am currently testing the app. Maybe, for more engagement, start processing cards faster just after installation. It feels weird to have to wait 30 minutes, just show me at least a card. Like the fact that I am installing DayFlow would be a positive experience.
Compliments for the Wizard - that one works perfect at least with Gemini. One little detail: You have a Github Star button in it, that really was at a non-logical place and made me think.
Recall (and Rewind) are similar in the sense that they both use screen data, but it's designed for retrieving specific things you saw, not semantically summarizing your time. My opinion is that they're completely different feature sets.
Which are wildly different when comparing a third-party hosted product (i.e., Microsoft), and a self-hosted OSS application that can use a self-hosted Ollama model.
There really isn't a reason that the screen data, once you have it, can't be used for more than one thing. I would guess that there isn't a whole lot stopping Windows Recall from doing very similar things.
Cute serendipity, rule of three. Neat project too; conceptually it sounds like an amazing ability to be able to better watch ourselves. Doing it via screenshots & AI feels like a fun sense-making adventure that actually makes a lot of sense, that can maybe try to pick through & discern what the screen is doing in a lot of different scenarios.
Given that ScreenMemory [1], an app that automatically records screenshots, has already saved my screen at intervals, it would be great for it to also have this AI summarizing ability.
I would imagine this could be one of the inputs along with a STT system as context to an LLM. Because in general we can speak faster than we can write/type and for me, specifically, after a point in the day typing creates a higher cognitive load than speaking.
1. "Create a reminder for reading this email at 5:00 pm" and this could infer what to do from the screen shot's description(plus a local MCP tool for calendar)
2. "Can you fetch that file form that project in that workspace and implement the pattern in the code on my vscode terminal?" It can lower cognitive fatigue of typing and clicking a bunch of place.
3. Take notes as I describe something on the screen. It could be for prompt composition e.g. get the link from my browser and the file on vscode and write code that does XYZ.
I've been using today and it seems like it's using 1 euro of credit per hour, is this normal ? Seems a bit expensive. I'm not running the trial of Gemini anymore. Would be nice to detect when there's no mouse movements / keyboard activity and stop recording in those case. And also stop recording when a media player is at fullscreen.
Couldn't we get a low-res version of this info by tracking the active window using a cli tool? For linux, there are several options. Not sure about Mac.
Another approach is to run OCR on 1FPS screenshots. Everything runs locally without draining the battery like an LLM would.
Funny enough, I had a similar idea a few weeks back; I jotted it down in my idea sketchpad. It felt a bit ambitious for an open-source side project, and I wasn't sure if it could even work with a local LLM. I was genuinely excited about it, nonetheless.
Now that I know it's totally viable, I've got even more reasons to build a Linux version myself.
Also you stated, and true, that there is much focus needed on improving your core feature.
It might be interesting to allow some kind of API / plugin area. So that people can expand on your core feature and add the desired parts. Might in the future expand to some kind of AppStore like feature with plugins.
That would keep your work focused and allows others to make it complete in their vision, and for others.
This would be helpful also for companies? Hence, ethically point of view it would violate the employee time in screen so there would arise issues with employees rights and HR?
Wow this is awesome! Wish I could try this on Windows. This is genuinely one of few time tracking solutions that piqued my interest. For now, I'll stick to manual labeling activities with my custom, simple tool: https://github.com/Klaster1/timer-5
Very cool idea. When using the Gemini option, what kind of cost would be expected to be incurred? I'd be satisfied by knowing the approximate number of tokens one would expect to be consumed by processing an hour of these recordings, and which specific model is being used.
Gemini 2.5 Pro is pretty expensive, mostly because videos take up a lot of tokens. It's roughly 1 million input tokens/hr, with a relatively insignificant amount of output tokens. Fortunately, Gemini has a very generous free tier, which is more than enough to cover daily usage. If you set up one paid project (and just don't consume any tokens), you can still use the free tier on a different project, and they can't train on your data.
Would be helpful to have a screenshot test tool alongside the api test tool. The app didn't create an application support folder yet. Possibly not enough time has passed but would be great to be able to troubleshoot sooner.
Thanks, yeah I do need to flesh out the debugging options. In the menu bar you can click the Dayflow icon which should allow you to view the recordings folder. The sqlite db is in that folder too if you want to poke around there as well.
Thank you! I can see that now. Checked, I have recordings. The SQLite seems to have very little data, despite recordings being present, and no cards show up in app. Possibly an LM Studio issue. I'll fiddle around with it and send an email if I can't get it working. Will test with Ollama in case there's some LMStudio error the API tester isn't catching.
This is the error I got: reason: No valid observations generated from frame fallback
I though about having something like this! This can be a great tool!
For engineers this can be a great tool to summarize the standup update, or even to recall what did we do yesterday
I'll check it out now
Love the idea, how you present it here and in the product. Clear, trustworthy and calming. Just installed and looking forward to try out. Half-random question: how has your way to the current UI / UX (visually and copy)?
It would be useful for freelancers if Dayflow automatically detected the client they were working for, to count hours spent on client, similar to what toggl.com does but automated
I have installed this and configured the API key, it's been three hours nothing is happening for some reason. The app doesn't show anything. Is it because I have a multi monitor setup?
Yep! Have tested it out on Qwen 2.5VL 3B and it works reasonably well on my 16GB Macbook Air. The only thing I will say is that I don't think it's a great idea to run local models on laptop battery, since it's quite compute intensive and drains kinda quickly. Have tested with Ollama and LMStudio, but you should be able to use any OpenAI compatible local server.
Would it be possible to check for the power adapter and run processing then? These are the types of things I've been thinking about for my own app: https://stardateapp.com
> Gemini leverages native video understanding for direct analysis, while Local models reconstruct understanding from individual frame descriptions - resulting in dramatically different processing complexity.
For people like me who haven't dabbled much with AI video processing and have no intuition for it, could you clarify the drawbacks of such a local-only approach vs what Gemini offers? I don't mean the performance or power/battery impact (that part is clear), just in terms of end-result and quality what the practical differences are.
I'm in the only-100%-offline camp here but would like to know what I'm missing out on since I won't even try Gemini here.
That would be really cool, but for the foreseeable future there's still a lot of room to improve how screen data is used so I'll mostly be focused on that.
Am I the only person that sees "AI" and "screen capture" and thinks no fucking way? I switched to Linux specifically to get away from data collection, why on earth would anyone want to opt into it?
The app has a local only mode. That's just your computer chip/gpu running a language model locally. It works even if all outgoing connections are blocked, as far as I can tell. What's the threat you're worried about in that scenario?
You should sell this to Lawyers and other professionals who bill per hour to reconstruct their billables for the day without missing anything. They would pay big money for something that recovered forgotten(unbilled) work throughout the day.
Devil's advocate opinion - this would also show how little per hour lawyers spend working :)
One of my first ever bug reports, was a submission to a company that made legal software.
In particular, it was a document management system built as a plugin for MS Outlook. (ew)
Most users, had no issue. However for one user, a lawyer, in particular, she would open and close a bunch of documents (using the built in pdf viewer) and then the application would crash, taking outlook with it, often requiring a restart.
I went over to view the behavior, and she was some kind of robot. Unlike her peers, she had 12 documents open at once, and she could update and bill (in minimum 6 or 7 minute increments) 12 customers cases in 15 minutes. It was like meeting the Usain Bolt of law practitioners. My back of the napkin math is that she billed like 3-4 hours for every hour she was online.
Open Email
Load Attachment
Review Attachment
Reply to email
Assign Email thread to case number
Close attachment.
12 times in 15 minutes.
The bug was that, after loading ~6 pdfs, the application would back off and wait to deallocate the memory. It would then later, randomly decide to write to that memory when another pdf was loaded, and go kaput.
Just to replicate the issue, I had to close and reopen pdfs so quickly my hands hurt.
It took 3 revisions of the bug report to get the software company to accept it and resolve it. And even then I think the pdf limit just increased, before we submitted another report and had it resolved permanently.
On that note, the principal of another law firm I supported would require us to cleanse his personal laptop of porn themed golf games he had downloaded on a regular basis.
The impression I get is that, lawyers work but the work is just unevenly distributed.
Were they porn-themed golf games, or golf-themed porn games? This is an important distinction.
Digging deep in to my memory, I recall that the user had at least one instance of Strip Putt Putt. I cant conceive of how that answers your question but its the best I can do.
It’s never occurred to me that either existed but…rule 34, I guess.
Something like Strip-Golf? For every under par she (or he) takes off an item of clothing. For every over they put on one, ha.
Sounds like a game Epstein and Trump (and some "enigmas") could've played at Mar-A-Lago. With cheating, though.
I get the sentiment behind your comment but I have a few lawyers in the family and they work round the clock. They might be in meetings or pouring over documents all day that might not look like work to the average software engineer but trust me, they do work hard. And it's true for everyone - from junior interns to senior partners.
> They might be in meetings or pouring over documents all day
FWIW it’s “poring over” when reading carefully.
From Merriam-Webster
As a general rule I find lawyers are real honest about their hours, especially as you get to bigger firms.
Anybody have a different experience?
> Devil's advocate opinion
Not sure if the pun was intended, but I'm here for it
lawyers have minimum billable time, if they reply to an email in 1min they will still bill you 10-15min. Ask me how I know :(
Wouldn’t you? If I switch context and interrupt my flow to answer a question I’m losing at least 20 mins to regaining focus, why shouldn’t that be reflected in billing?
Knowledge work is knowledge work, no point belittling colleagues in a different profession.
Thats how MSPs operate too. At least the good ones. Billing increments are sometimes as low as 6 minutes, or as high as 30. 15 minutes is average in my experience.
I'm a litigation legal admin - I have been for 25-30 years. I instantly brought this up to an associate, telling them, "Maybe not now, but before you retire, this'll be the norm in the industry."
She had been complaining the day before about having to reconstruct a huge bunch of little 0.1 entries involving e-mails to various individuals in cases. If it could be done automatically, through a local LLM? chef's kiss
Trust me, law is definitely where you want to land this thing.
In all honesty, I have absolutely no negotiating power or decision-making authority for my firm, but it's a big one -- if that's a direction you want to go, can't guaranty I can swing enough weight, but I probably could find you the right people to talk to, give you an introduction.
I'll also have to add, though, that you'd have to figure out a way for it to be cross-platform or live outside just macOS. Unfortunately, that's a very uncommon choice in the legal world (or anywhere else).
I’m a software contractor and I’ve wanted this forever. I’m prototyping something on Linux now.
Let me know if this Linux app gets anywhere!
I've used https://github.com/ActivityWatch/activitywatch before on Linux, it's actually quite good but takes a fair bit of fiddling to get good results.
There's no plugin for screenshot or recording in 1fps.
Yeah, but it's written in a modular way and extending it is not as painful as one would expect. I actually went that way and wrote a couple of custom watchers for things like that.
+1! I was actually looking to fork ActivityWatch to support your exact flow.
Per hour? In the UK they bill by the 6-minute! If ever anything told you something about a profession...
They sell in blocks of 1/10th hours??Thats gotta be the jail call special
Every lawyer in the USA that I've ever worked with also bills in 6 minute increments. Which means every email is 6 minutes. Every phone call is at least 6 minutes. etc.
Isn't this better though? The alternative being that every email or phone call is an hour, or that they batch it up based on gut feel. If I get a phone call that could easily steal 6 minutes of focus time (note that I specify focus time, even if the call is only 30 seconds you'd have to mentally switch tasks back and forth).
I’m not sure that 6 minutes is a useful denomination of focus time. Maybe legal work is too different from IT and it makes sense there, but when I need to focus on something, 15 minutes is the smallest amount I would allocate for a task.
Really nice! I currently use ActivityWatch for tracking tasks on PC.
Some things I would like to be able to do with software like this:
- Identify the 'spark' of a distraction. For example, opening my email inbox to read a specific email also shows me many unrelated emails. These can easily be the cause of a 5-15 minute distraction. This information is often actionable. I installed browser plugins to hide my youtube suggested videos and my distractions went down. I made sure to close all unused windows to avoid catching a glimpse of unrelated work.
- Identify repeated tasks, and the cadence of those tasks. Do I manually make an invoice once a week for a particular edge case? Is the process basically identical every time. Could this be automated?
- How was I feeling before, during and after a task. (This is a very broad and intentionally not well-defined question, but I think it has the most promise for improving procrastination and task initiation).
Yep, helping people understand their distraction patterns would be an amazing feature. I find myself doing the same thing, funnily enough I also have that same Youtube extension.
I love the product concept but the fact this person has an almost empty github and suddenly launches an app that can easily be spyware concerns me a lot :). A lot of security concerns with password etc.
Could easily dismiss those concerns by looking at the source code instead of snooping around their profile, especially if you’re on GitHub anyway :)
Not saying the author has this intent, but if I was trying to spy on people, I wouldn't include spyware in the initial release...
I would not be comfortable sending my bank info passwords and all sorts of other sensitive data that I input and see on my screen to Gemini. How much is the qualitative performance difference with a local model?
If I had to put a grade on my own experience and evals, Gemini 2.5 pro produces A- results and qwen2.5vl is maybe like B-/C+. Obviously everything's nondetermistic, so it's hard to guarantee a level of quality.
I'm reading through papers that suggest it should be possible to get SOTA performance on local models via distillation, and that's what I'll experiment with next.
Any insights on qwen-3 omni yet?
Looks awesome, but a 30B model is too big. Vast majority of people probably have 32GB of RAM or less unfortunately.
Also, if your not using an enterprise edition of gemini where your data is not used for model training, your sensitive data prompts and responses is 100% available to google.
Google owns my email, browser, phone operating system, and a small amount of passwords. I assume that it has already stolen all my confidential data by now.
Your passwords should never be visible on screen anyway: They go straight from a password manager into a censored input field.
I haven't seen this mentioned, but I immediately thought this could be a great tool for folks with ADHD. The potential for seeing what kinds of things regularly trigger distraction (I know, everything, right?) and any patterns that exist (i.e. every time I make a git commit, I go check Hacker News and lose 15 minutes). As well as being able to review day that was captured automatically is huge. The best success I had with tracking what I did was when I used to use TimeRescue to ensure I had accurate record of hours for clients, but every attempt to use anything that requires manual entry fails very quickly (either too distracting everytime I use it, or I literally just forget to use it).
Going a step further, "real time" (given processing delay) to help stay on task when the focus has shifted to something unrelated (maybe allow the individual to define this or say yes/no to train the prompts as it goes).
Anyways, it looks great. I also liked the _idea_ of Windows Recall, so to see something like this that can be privacy first is really nice.
Pretty nice. How does it handle multiple displays? I've set it up with local Ollama, and it seems to only record and analyze one of my two secondary displays. It would be ideal if I can select which one is used if the recording is limited to a single display, or even better if it can record and analyze the entire multi-monitor desktop surface.
edit Nvm, it seems it always records the display that is currently in focus. That is probably the better way to handle it, since it automatically solves the "ignore what's shown but not interacted with on secondary displays" problem.
This was my question also, I think "even better if it can record and analyze the entire multi-monitor desktop surface" would be the best option. I don't know what the impact of that would be on both recording size and AI processing time, but just because one monitor is focused doesn't always mean what's happening on another is ignored. Some examples: an ongoing meeting or watching a video on one screen while taking notes on another; or coding on one screen and a browser/app auto-refreshing on another.
Yep, you figured out how it works! That was the easiest solution I could come up with. I'm sure theres additional context on other screens but this was a good 90/10 solution.
I see a potential issue where you're in a zoom call in one monitor and working in something else in the other (multitasking ) how to handle that ?
Similar Idea to screenpipe. That gives you more customization:
github.com/mediar-ai/screenpipe
screenpipe founder here, love to see more products in this area, ideally OSS, local, no lock in, API/MCP friendly
kind of sad it's macos only, i'm mostly windows user now :)
side effect of working for enterprise customers. congrats on the fund raise.
I'd only ever consider doing it with a local model, but this looks really cool!
Thanks! Between my friends and I, it's about a 50/50 split between local and cloud. I think it's great to be able to pick the tradeoff between quality/privacy based on your own privacy preferences.
Great project! I’ve had a similar experience with Rewind and the related privacy concerns. A quick thought: if I recall correctly, Rewind performs OCR locally, so it only needs to send textual data. Since you’re focusing on macOS, you could rely on VNRecognizeTextRequest and skip the extra OCR complexity. It might also help to detect and mask sensitive information with lightweight models (e.g., BERT), especially when leveraging cloud-based AI.
Woah didn't know about VNRecognizeTextRequest, that's super cool thanks for flagging!
On one hand I'm super enthusiastic about your project.
This could help battle procrastination, organize your time in a long run, bill your clients more efficiently, etc. 20 years younger, hyper productive me would kill for such product.
But then I recall when I accidently suggested TimeRescue to my boss at one time, and suddenly he was skimming though everyones daily logs to see if they're spending 100% of their times in business facing apps.
When I first heard about "covid mouse mover devices" that faked activity for remote workers I thought it was a joke. Seriously.
But I'm afraid this is the dystopian future. Employers constantly looking at your screen and getting spreadsheets with your daily effort.
Overall, very disturbing product.
This was my first thought too. The last generation of activity tracking, while still dystopian, was a little different at least in that it was mainly statistical. So action-wise, it might point managers at "potential problems," but doesn't make its way into a performance review (e.g. "your mouse only traveled 81.72 screen-miles this quarter, 2 standard deviations below the mean, while you also scored the lowest on number of keystrokes with vscode as the active window..."). If a manager really wanted to summarize exactly what was done they had to spend an almost equal amount of time to watch. To some degree, this alleviates that.
Yea, honestly I would hate if people used this to track _other_ people, especially bosses. I wanted to build something that gave people more agency to do more with their precious time, but there definitely is a fine line here.
Probably someone long ago said the same about hope for pointy sticks not being used just for hunting animals. Yet someone will likely make a pointy stick whether you do it or not.
oppenheimer said that about dynamite /s
Woah this is fab; much less cognitive load than manually using a time tracker. And I'm glad that there's a local option and a "BYO key" option for privacy!
Feel like something of this shape should have existed for a while, but this is very well executed!
if you are on a Zoom/video call, does anyone know if you would have to declare that your "recording" it? I'm thinking more from the legal perspective of wiretapping/consent laws. If you have live transcripts/subtitles does that change any legal requirement.
Yes, in my state it's generally illegal to take a screenshot of a zoom call without announcing you are recording the conversation. I'm not certain, but I think the issue is the storage of the 1fps video, not the AI summary.
For those expecting something more along the lines of a `git log` sort of thing, like a command line tool, there's `doing`.
https://brettterpstra.com/projects/doing/
Who doesn't love Windows Recall
Im confused how much loves this finds here while Recall was rightfully critisised. ITS the Same Pictures
I am currently testing the app. Maybe, for more engagement, start processing cards faster just after installation. It feels weird to have to wait 30 minutes, just show me at least a card. Like the fact that I am installing DayFlow would be a positive experience.
Compliments for the Wizard - that one works perfect at least with Gemini. One little detail: You have a Github Star button in it, that really was at a non-logical place and made me think.
wait... isnt this pretty much what Microsoft was doing with Recall?
Recall (and Rewind) are similar in the sense that they both use screen data, but it's designed for retrieving specific things you saw, not semantically summarizing your time. My opinion is that they're completely different feature sets.
The backlash for Recall was not based on the feature set, it was because of the massive privacy and security concerns.
Which are wildly different when comparing a third-party hosted product (i.e., Microsoft), and a self-hosted OSS application that can use a self-hosted Ollama model.
The feature isn't the problem.
There really isn't a reason that the screen data, once you have it, can't be used for more than one thing. I would guess that there isn't a whole lot stopping Windows Recall from doing very similar things.
It's somewhat related two other recent submissions,
Replace PostgreSQL with Git for your next project for git data storing. https://news.ycombinator.com/item?id=4535144 https://devcenter.upsun.com/posts/why-you-should-replace-pos...
Consumer.today day-logging single user microsite. https://consumed.today/ https://news.ycombinator.com/item?id=45351446
Cute serendipity, rule of three. Neat project too; conceptually it sounds like an amazing ability to be able to better watch ourselves. Doing it via screenshots & AI feels like a fun sense-making adventure that actually makes a lot of sense, that can maybe try to pick through & discern what the screen is doing in a lot of different scenarios.
"Records screen at 1 FPS in 15-second chunks.“
If it’s recording 15 seconds, how often are you doing that? Once every 15m as the analysis interval is 15m?
Looks like it's recording all the time and analyzes 900 screenshots every 15 minutes? And it keeps records for 3 days.
So I'm not sure I buy the lightweight/low-impact claim.
Given that ScreenMemory [1], an app that automatically records screenshots, has already saved my screen at intervals, it would be great for it to also have this AI summarizing ability.
[1]: https://screenmemory.app/
I would imagine this could be one of the inputs along with a STT system as context to an LLM. Because in general we can speak faster than we can write/type and for me, specifically, after a point in the day typing creates a higher cognitive load than speaking.
1. "Create a reminder for reading this email at 5:00 pm" and this could infer what to do from the screen shot's description(plus a local MCP tool for calendar)
2. "Can you fetch that file form that project in that workspace and implement the pattern in the code on my vscode terminal?" It can lower cognitive fatigue of typing and clicking a bunch of place.
3. Take notes as I describe something on the screen. It could be for prompt composition e.g. get the link from my browser and the file on vscode and write code that does XYZ.
I've been using today and it seems like it's using 1 euro of credit per hour, is this normal ? Seems a bit expensive. I'm not running the trial of Gemini anymore. Would be nice to detect when there's no mouse movements / keyboard activity and stop recording in those case. And also stop recording when a media player is at fullscreen.
Couldn't we get a low-res version of this info by tracking the active window using a cli tool? For linux, there are several options. Not sure about Mac.
Another approach is to run OCR on 1FPS screenshots. Everything runs locally without draining the battery like an LLM would.
You definitely could! I think it would just be harder to get good semantic understanding of what you did during a segment of time without LLMs.
This is amazing and yet I think it would need an existential angst mode to capture those days when I am doing video calls with various teams all day.
Maybe patching https://github.com/JerryZLiu/Dayflow/blob/main/Dayflow/Dayfl... to say "Describe what you seen in this computer screen in the style of Werner Herzog" would do it...
Nicely done.
Funny enough, I had a similar idea a few weeks back; I jotted it down in my idea sketchpad. It felt a bit ambitious for an open-source side project, and I wasn't sure if it could even work with a local LLM. I was genuinely excited about it, nonetheless.
Now that I know it's totally viable, I've got even more reasons to build a Linux version myself.
Really cool!
As already seen in the comments there are lots of desires to add more data compared to just screen input.
Could be things like:
- Apple HealthKit / watch - custom apps - Phone logs
Also you stated, and true, that there is much focus needed on improving your core feature.
It might be interesting to allow some kind of API / plugin area. So that people can expand on your core feature and add the desired parts. Might in the future expand to some kind of AppStore like feature with plugins.
That would keep your work focused and allows others to make it complete in their vision, and for others.
This would be helpful also for companies? Hence, ethically point of view it would violate the employee time in screen so there would arise issues with employees rights and HR?
Looks like 98% of my day is Hacker News. This thing must have a bug.
Right?
I think this is pretty cool but I spend most of my day on a laptop I don’t own and there’s no way I could get this on there.
This reminds me of Stephen Wolfram's efforts to analyze his life: https://writings.stephenwolfram.com/2012/03/the-personal-ana...
Wow this is awesome! Wish I could try this on Windows. This is genuinely one of few time tracking solutions that piqued my interest. For now, I'll stick to manual labeling activities with my custom, simple tool: https://github.com/Klaster1/timer-5
If you're on Windows, give this a try: https://donethat.ai - cloud-based but you can use your own gemini key
my custom simple tool is a stopwatch on a phone ¯\_(ツ)_/¯
This is super rad. Love it being Open Source, and with the option to choose local models. You’re awesome, thanks!
Testing it today, because my Mac thinks I'm sharing my screen, it's suppressing notifications on phone, mac, etc. Need to figure out how to fix that.
Curious how this works with multi monitor setups, e.g. watching a viedeo while researching travel plans.
Very cool idea. When using the Gemini option, what kind of cost would be expected to be incurred? I'd be satisfied by knowing the approximate number of tokens one would expect to be consumed by processing an hour of these recordings, and which specific model is being used.
Gemini 2.5 Pro is pretty expensive, mostly because videos take up a lot of tokens. It's roughly 1 million input tokens/hr, with a relatively insignificant amount of output tokens. Fortunately, Gemini has a very generous free tier, which is more than enough to cover daily usage. If you set up one paid project (and just don't consume any tokens), you can still use the free tier on a different project, and they can't train on your data.
Would be helpful to have a screenshot test tool alongside the api test tool. The app didn't create an application support folder yet. Possibly not enough time has passed but would be great to be able to troubleshoot sooner.
Thanks, yeah I do need to flesh out the debugging options. In the menu bar you can click the Dayflow icon which should allow you to view the recordings folder. The sqlite db is in that folder too if you want to poke around there as well.
Thank you! I can see that now. Checked, I have recordings. The SQLite seems to have very little data, despite recordings being present, and no cards show up in app. Possibly an LM Studio issue. I'll fiddle around with it and send an email if I can't get it working. Will test with Ollama in case there's some LMStudio error the API tester isn't catching.
This is the error I got: reason: No valid observations generated from frame fallback
I though about having something like this! This can be a great tool! For engineers this can be a great tool to summarize the standup update, or even to recall what did we do yesterday I'll check it out now
Love the idea, how you present it here and in the product. Clear, trustworthy and calming. Just installed and looking forward to try out. Half-random question: how has your way to the current UI / UX (visually and copy)?
It would be useful for freelancers if Dayflow automatically detected the client they were working for, to count hours spent on client, similar to what toggl.com does but automated
I have installed this and configured the API key, it's been three hours nothing is happening for some reason. The app doesn't show anything. Is it because I have a multi monitor setup?
Congrats on a nice looking app that will be very useful for individuals (though potentially misused by toxic managers).
Kudos particularly for the efforts you've gone to on explaining privacy implications.
Thanks! Wanted to build something I'd personally be comfortable using.
What makes this similar to "git log", other than it show events happening in an order? It looks more like my calendar layout than git log to me
The onboarding flow is neat. I will give it a try over the next days. The local setup makes the computer heat a bit every 15 minutes, but it's ok
You seem to have a naming conflict with https://dayflow.ai/
Maybe both asked the same LLM for inspiration
It’s nice to forget everything you’re working on periodically and examine the pieces of what you’ve built and redecide what they mean if anything.
Nice work, does this work with local (100% offline) models assuming you have decent hardware and are serving them up with llama.cpp or similar?
Yep! Have tested it out on Qwen 2.5VL 3B and it works reasonably well on my 16GB Macbook Air. The only thing I will say is that I don't think it's a great idea to run local models on laptop battery, since it's quite compute intensive and drains kinda quickly. Have tested with Ollama and LMStudio, but you should be able to use any OpenAI compatible local server.
Would it be possible to check for the power adapter and run processing then? These are the types of things I've been thinking about for my own app: https://stardateapp.com
Wow, yeah that's clever I hadn't thought of that. Will add as an advanced setting.
You've mentioned in the docs that:
> Gemini leverages native video understanding for direct analysis, while Local models reconstruct understanding from individual frame descriptions - resulting in dramatically different processing complexity.
For people like me who haven't dabbled much with AI video processing and have no intuition for it, could you clarify the drawbacks of such a local-only approach vs what Gemini offers? I don't mean the performance or power/battery impact (that part is clear), just in terms of end-result and quality what the practical differences are.
I'm in the only-100%-offline camp here but would like to know what I'm missing out on since I won't even try Gemini here.
Nice that's great. I have a 96GB M2 Max that's plugged in 99.9% of the time so that's not an issue. Cheers for the response!
Is it possible to include wearables as a data sources?
i.e. apple watch for sleep, running, activity levels? it could really give a 360 view of your life
That would be really cool, but for the foreseeable future there's still a lot of room to improve how screen data is used so I'll mostly be focused on that.
I'm sure my employer would be thrilled with a background process taking screenshots every second
Congrats. This is very well executed.
This is amazing - just the tool I needed; thank you so much!
Great! This is what I want for long! And UI is good too.
Wow such a great idea. Will you monetize this ?
this is very cool - thanks for sharing!
I need this so badly - but on Linux :)
the fact that people already built this with open source multi modal model is astonishing
Very nice. Beautiful UX.
this is amazing! thanks for doing this!
Am I the only person that sees "AI" and "screen capture" and thinks no fucking way? I switched to Linux specifically to get away from data collection, why on earth would anyone want to opt into it?
The app has a local only mode. That's just your computer chip/gpu running a language model locally. It works even if all outgoing connections are blocked, as far as I can tell. What's the threat you're worried about in that scenario?
Fair enough, I didn't see that part. I'm still not interested but that seems like a far more reasonable option.
What kind of problem is this solving? Why would I install spyware on my machine? lmao
wow
just wow
2025 is getting surreal online