Momentic launches AI QA agent that autonomously tests apps for bugs
YC-backed startup releases Mo, an AI agent for end-to-end testing across web and mobile platforms, claiming 5x bug detection over Claude Code.
What to know
- Momentic launched Mo, an AI QA agent that autonomously tests web and mobile apps, claiming 84% precision and highest recall against competitors in a 1,396-bug benchmark, though at higher per-bug cost ($2.67 vs $0.80 for Codex).
- Early technical community skepticism centers on closed-source, cloud-only architecture versus open-source alternatives; token-based pricing model; and whether performance gains justify the cost over running Claude models with custom harnesses.
- Users report onboarding friction, iOS compatibility errors, and UI inconsistencies in the dashboard, while also raising questions about real-device mobile testing roadmap and silent test failures (the riskier failure mode than precision/recall metrics capture).
- Co-founders defend speed through browser/device fleet parallelization and cite customer testimonials of 3-6x faster test writing, but community debate remains unresolved on whether the service differentiates sufficiently from self-hosted open-source tools or direct LLM use.
The dispute Whether Momentic's cloud-based parallelized testing and integration workflow justify closed-source, token-metered SaaS pricing versus open-source self-hosted tools or direct LLM API use with custom harnesses. · positions read across 40 posts and comments
Momentic's speed and bug-finding edge justify the service despite pricing; parallelized testing and team integration outweigh self-hosted alternatives.
-
“Our team's experience with Momentic has been great. Setting up tests is simple and the developer experience has been fantastic.”
Early user · Momentic website ↗
Closed-source, cloud-only proprietary tools with token pricing are not worth the lock-in; open-source alternatives or direct LLM use are preferable.
-
“Closed source, non self-hosted in 2026? Nah. What do you guys have that's better than a self-hosted, open source extensible QA AI tool like Mira?”
upmostly · Hacker News ↗
The benchmark metrics (precision/recall) don't measure the most dangerous failure: silent passes on broken flows; mutation testing is a more rigorous benchmark.
-
“Not sure precision/recall is the number I'd look at first, because the thing I'd actually want to know is how often the tool says pass on a flow that is already broken, and a silent pass is worse than a flaky test, since the flaky one at…”
daniiel · Hacker News ↗
Technical issues with onboarding, iOS compatibility, UI inconsistencies, and cloud-based slowness undermine the product experience.
-
“I tried to use mo on the platform (I have been wanting to use momentic for a while). There was no onboarding really, now it's trapped in me needing to configure auth but I don't have time to click through 20 pages to find out how to do it…”
ecares · Hacker News ↗
Wei-Wei Momentic co-founderJeff Momentic co-founderHenry Momentic engineerGabe (MGabrielDenton) Momentic engineer
How it unfolded 12 developments, newest first · click a bar or a number to jump articlespostscomments
-
12
Commenter requests benchmarks with codebase access, citing agent efficiency gains
A user with prior experience at another company suggested Momentic benchmark scenarios where agents have read access to source code, noting this setup produced faster debugging and code correlation without benchmarking overhead.
-
Good to see Momentic's demo. We at Vostride are building something similar, but we feel the verification layer should be open for verification, too.We built agent-qa (https://github.com/vostride/agent-qa), an open-source self-improving QA agent. The harness becomes outdated every 3 to 6 months. Unfortunately, the big players are unable to maneuver…
2 more of the top 3 · 21 posts in this stretch
-
I drive a real browser against websites as my job, so I've been on the buyer side of this. What decides whether I keep an agent in the loop isn't how many bugs it finds, it's whether I trust the red result without redoing the check myself.Two things bit me early. Agent written assertions drift to whatever is stable to assert: page copy, a fixed…
-
The haukebri point deserves more weight than it's getting: for a professional tester, recall is not the metric — trust is. One noisy finding per run and the report gets ignored, which means every real finding in it dies too. A tool that finds 3x more bugs at the cost of reader trust has negative value, because the reader's attention is the scarce…
-
-
11
User dismisses Momentic as too slow and expensive compared to Claude and Maestro
One early adopter described cloud-based architecture as too slow and pricing as prohibitive for developers on subscription services, favoring Claude with basic prompts or Maestro scripts instead.
“it's too slow as it relies on the cloud and it's way too expensive since most developers use a subscription service…”
— AgentMasterRace -
I think there is something here with the DAG exploration graph that systematically increase the surface area of exploration when the agents QA's an app. And I've definitely noticed in my own experience that Playwright QA steps in a coding agent will catch certain bug classes very reliably but some edge cases will slip through from time to time. So…
2 more of the top 3 · 4 posts in this stretch
-
Hey there, it's Jeff, one of the co-founders. Sorry you had a bad experience. Would you mind sending a report to support@momentic.ai? We look at each report closely.We parallelize testing across a massive fleet of browsers/devices so it should actually be much faster than Claude. The open source benchmarks prove it :)Actually a lot of our mobile…
-
I used this service very briefly, but it's too slow as it relies on the cloud and it's way too expensive since most developers use a subscription service.Even just Claude with basic prompts is better, but even better with skills.. and then even better with maestro scripts.
-
-
10
User flags UI inconsistency in Momentic dashboard during bug detection
Early tester reported confusing discrepancy: 'No bugs yet' message in one window while Bugs tab showed 11 bugs; after run completed, dashboard said 'Mo didn't find any bugs' while Bugs tab showed 12 bugs.
-
Congratulations on the launch! It looks interesting. I ran a check on one of the pages of my site. Here are a few points that caught my eye -hopefully, you’ll find them useful:- It’s a bit confusing: the window below "Live Mo's screen" says "No bugs yet," while the tabs at the top (Agents / Bugs / Mo's screen) indicate "Bugs: 11." Once the process…
-
-
9
Technical expert argues precision/recall misses key metric: silent failures
A developer building a competing QA tool noted that recall and precision don't capture the most important failure mode—silent passes on broken flows—and recommended mutation testing (deliberately planting bugs) as a better benchmark.
“a silent pass is worse than a flaky test, since the flaky one at least makes some noise…”
— daniiel -
Not sure precision/recall is the number I'd look at first, because the thing I'd actually want to know is how often the tool says pass on a flow that is already broken, and a silent pass is worse than a flaky test, since the flaky one at least makes some noise.I'm building something in the same space and this bit me quite early, so what I ended up…
1 more of the top 2 · 2 posts in this stretch
-
Hey thanks for sharing your work! Just wondering how many of the 1396 came from Mo, and what happens to a bug that Mo can't reproduce?
-
-
8
User reports Slack integration automatically enabled without clear consent
A commenter discovered Momentic auto-enabled Slack notifications and shared their email with Slack upon login, though settings allowed disabling it.
“they automatically hand over your email to Slack to contact you upon login even tho I don't remember this being mentioned anywhere at all…”
— vfosnar -
> Momentic invited you to work togetherso apparently they automatically hand over your email to Slack to contact you upon login even tho I don't remember this being mentioned anywhere at all. I went to check the Momentic settings and theres a Notifications section with Slack turned off...used my public email so idc but others may
-
-
7
Momentic co-founder Jeff responds to performance criticism and Maestro comparison
Jeff, co-founder, addressed user concerns about speed and cost, claiming parallelization across massive browser/device fleets makes Mo faster than Claude, and noting many Maestro users switched to Momentic due to maintenance burden.
“We parallelize testing across a massive fleet of browsers/devices so it should actually be much faster than Claude. The open source benchmarks prove it :)…”
— jeffan -
I tried to use mo on the platform (I have been wanting to use momentic for a while). There was no onboarding really, now it's trapped in me needing to configure auth but I don't have time to click through 20 pages to find out how to do that :( I really wanted to like you!
-
-
6
Community members cite privacy concerns and token pricing model
One user objected to token-based pricing as an 'instant turn off' in the subscription ecosystem, and another reported that Momentic automatically handed over their email to Slack without clear notification.
“So is this yet another SaaS product that's going to attempt to charge me for token usage? Sorry to sound negative, but nowadays it's an instant turn off because it's like the current subscription crisis on steroids.”
— aliasxneo -
Hey! I'm Henry, one of the engineers working on Mo. To answer your questions:- I expect there will be many more competitors to come, as there already are. I also think achieving these performance improvements is non-trivial and depends on a lot of internal tooling we've built over the last two years (see next answer). As with any software, there…
1 more of the top 2 · 2 posts in this stretch
-
Closed source, non self-hosted in 2026? Nah.What do you guys have that's better than a self-hosted, open source extensible QA AI tool like Mira?[1]
-
-
5
Skeptics question competitive durability and mobile device roadmap
A commenter questioned whether Momentic expects no open-source competitor will eventually match performance, and pressed for details on real-device mobile testing, calling that 'the real edge here.'
-
Questions:- I would always pick an open source tool I can strap to my custom harness over using a service like this. Do you somehow expect that no such competitor will arrive that performs as well?- If you are YC 2024 what have you been doing for 2 years?- The real edge here would be supporting mobile development builds on real devices, is this on…
-
-
4
Users report technical issues with iOS compatibility and onboarding friction
Early testers encountered specific errors with iOS app testing (incompatibility with hosted simulator) and complained about missing onboarding documentation. One user stated configuration required clicking through 20 pages.
“The selected XXX build (com.XX.XX) cannot be installed because it is incompatible with the hosted iOS simulator.”
— brahyam -
tried testing it with my ios app but got: "The selected XXX build (com.XX.XX) cannot be installed because it is incompatible with the hosted iOS simulator. No XXX behavior could be tested."would be great to have more details about whats not compatible, I assume iOS version? app requires 27
-
-
3
Users question cost-benefit versus running Claude models with custom harness
Multiple commenters challenged whether Momentic justifies its pricing against simpler alternatives like running Claude Opus 5.5 directly with a custom Playwright harness using existing model subscriptions.
-
> Mo is a custom agent harness for catching bugs that builds on the past 3 years of Momentic and millions of test traces from complex, real-world applications. Mo is built for recall: it finds the most real bugs, though Codex is a bit more precise at 86% vs 84%. The tradeoff is cost: Mo costs $2.67 per bug vs $0.80 for Codex, less than Claude Code…
2 more of the top 3 · 4 posts in this stretch
-
The last 3 years saw tons of "custom agent harnesses" come up, only for them to be superseded by the next generation of frontier models, which could do the same (and more) in their generic harness. So, what will you sell when Opus 6.0 swallows Mo's edge in 5 months from now? And probably cheaper, when using a Claude subscription instead of paying…
-
Hey I'm Gabe (one of the engineers @Momentic). In general Mo's advantage is in the total amount of bugs found. In comparison to claude code we find roughly 5x as many bugs at 35% less per bug (for the 60 min session).
-
-
2
Early users express concerns about vendor lock-in and open-source alternatives
Technical community members questioned whether Momentic's closed-source, cloud-only model justifies its cost and complexity compared to self-hosted or open-source QA tools. One commenter asked whether the company expects no competitive open-source tool will match performance; another referenced a competitor product called Mira.
“Closed source, non self-hosted in 2026? Nah. What do you guys have that's better than a self-hosted, open source extensible QA AI tool like Mira?”
— upmostly -
Congrats on the launch!Some questions are top of mind after having done POCs and trials with other companies that have products in this space (excuse me if they are answered somewhere obvious - I skimmed the site and your post before drafting this):1. What is vendor lock-in like with Momentic? In my mind there's space for a mature QA platform that…
1 more of the top 2 · 2 posts in this stretch
-
So is this yet another SaaS product that's going to attempt to charge me for token usage? Sorry to sound negative, but nowadays it's an instant turn off because it's like the current subscription crisis on steroids. BYOA should be a default.To be more constructive: let's say I have 2 frontier subscriptions and I end up with 20% left on each at the…
-
-
1
Momentic releases QA Bench benchmarking Mo against Codex, Claude Code, Devin
The company published benchmark results comparing Mo against three competing QA agents on 1,396 labeled bugs. Mo achieved 84% precision with highest recall, while Codex scored 86% precision. Mo's cost was $2.67 per bug found, versus $0.80 for Codex, $9.00 for Claude Code, and $7.35 for Devin.
“Coding agents have made it dramatically faster to build software. The bottleneck is now knowing it works. Verification belongs in the agent loop.”
— Momentic founders, Co-founders · source -
background
Momentic founders launch Mo AI QA agent on Hacker News — Wei-Wei and Jeff, both software engineers from dev tooling backgrounds at Robinhood, Retool, WeWork, and Qualtrics, announced Mo, an AI QA agent that autonomously tests web, iOS, Android, and soon desktop applications. The system writes test flows in plain-English YAML and can repair tests when UI changes occur.
What people are saying 10 voices from 1 site · best of 40 · verbatim
- Will Momentic's performance advantage persist once open-source competitors mature, and what proprietary advantage makes lock-in worthwhile?
- Do the benchmark metrics (precision/recall) adequately capture the worst-case failure mode—silent passes on broken flows—or does mutation testing reveal a different picture?
- Can Momentic offer real-device iOS/Android testing (not just simulators), and what's the roadmap timeline?
- Nov 6
-
Cool benchmark setup, but I’d sanity-check the evaluation a bit more before trusting the $/bug and precision/recall comparisons.In particular: how is “bug” defined and de-duplicated across agents/runs? Are the 1,396 human-labeled bugs the same ones used for judging (i.e., any leakage via traces/dataset construction)? And since precision is partly…
- Nov 5
-
That's where UI-only testing gets tricky. A flow can look fine even when the request returns 200 with an error in the body, or a background job gets accepted but never actually completes.Seeing network requests helps, but for async flows you'd also need some way to verify the final application state. Does Mo have access to that kind of…
- Nov 4
-
The benchmark is the part I would push on, and I say that from assessment work. A test built by the people selling the tool, with bugs labelled by the same team, will usually flatter the tool, not because anyone cheated, but because the cases reflect what the builders already think counts as a bug.Planted and labelled bugs are also cleaner than…
- Nov 3
-
One thing I've been thinking about as coding agents take on more implementation work is the difference between finding bugs and verifying that the right thing was built.An app can work perfectly from a functional standpoint while still deviating from the original requirements or architectural decisions. We've seen agents quietly simplify…
-
The recording and steps part is what caught my eye. A bug report with just a description usually turns into a bunch of back and forth before anyone can reproduce it, so having the actual steps there sounds like it would save a lot of that. I am curious how that holds up on the borderline ones though. When a report is not clearly a bug, does that…
-
Hey there! Thanks for watching our demo! Would love to run Mo on some other, public, benchmarks and share results. I'm curious how you're evaluating your framework's ability to find bugs vs just execute flows in an app? I also took a look at your site and it seems like you're product is more akin to Momentic regression tests, as opposed to Mo, as…
-
One metric I'd be interested in is defects discovered per hour of test maintenance. Traditional end-to-end tests can become expensive when interfaces change frequently.For AI-driven testing, I'd want to compare:1. Real defects discovered before release 2. False positives requiring investigation 3. Maintenance effort after UI changes 4. Bugs that…
-
The point about LLMs reinforcing their own assumptions matches something I keep running into: when the same model writes the code and the tests, the tests tend to encode the implementation, so they pass by construction. What has helped more than a better model is separating the roles, e.g. tests written from the spec or ticket before the code…
-
Funny seeing Saleor in your benchmarks. I was digging through that codebase recently for something else and ran into a nasty bug along those same lines.They have a price recalculation task meant to only run on dirty listings, except someone forgot to assign the filtered queryset back to the variable. So it just recomputed every single listing on…
-
I'm curious to see benchmarks when you have access to source code. While traditional QA does not get access to the codebase, with LLMs being primary authors of the code, there are no real practical barriers to giving code access to the testing agents.At my last job, we did this - gave access to codebase to codex/CC, allowed it to drive the app via…