Product Validation Methods

Rick Rubin went on stage in Helsinki the day after my talk. Someone asked how he resolves creative differences with artists. His answer was simple: change the conversation from "I disagree" to "let's build it." Then he shared a story: An artist played him a song. The transition didn't work. Rubin told him so. The artist said, "We'll just cut that part in half." Rubin thought to himself: What a dumb idea. But he didn't say that. He said, "Let's try it." The artist played it. It worked. Rubin is a legend. He's produced everyone from Johnny Cash to Jay-Z. Instead, he bit his tongue and let the artist prove him wrong. The principle: when you make an idea tangible, it stops being the person's idea. It becomes something you can both look at objectively and improve together. Once you build it, the truth is obvious. Here's what this looks like in practice: Your designer wants to change the entire homepage layout. You think it's too risky. Instead of three meetings debating it, you say: "Let's build a prototype and test it with 50 users this week." Your sales team wants to restructure the pricing page. Instead of blocking it because you're worried about conversions, you say: "Let's run it as an A/B test on 20% of traffic for two weeks." Your engineer wants to rebuild a core feature from scratch. You think it's overengineered. But instead of killing it in the planning phase, you say: "Spike it out in three days and show me if the performance gain is real." You're not saying yes to everything. You're saying, "Let's find out." Rubin also said something that stuck with me: "If there's disagreement, I always side with the artist's vision. Because to them, it's their career. To me, it's just one piece of my portfolio." Most leaders think backing down makes them look weak. Rubin knows that siding with the person who has the most at stake makes better work happen. Your job isn't to be right. It's to create the conditions where the best idea wins. Stop debating. Start building. P.S. This insight is from this week's newsletter where I break down why Yamaha dominates while Steinway got sold to private equity: https://lnkd.in/efSqP_9K P.P.S. Access additional research links, the podcast, and the full archive in the first comment 👇 Thank you to Nordic Business Forum!

One of the biggest mistakes in growth is confusing “people use it” with “people love it.” So at Lovable, we built a composite qualitative metric to measure actual product love. The Lovable Score is a combination of: Net Promoter Score | NPS -> How likely are you to recommend? Sean Ellis PMF -> How would you feel if you could no longer use us? CSAT -> How satisfied are you? CES -> How easy was it to use? Any one of these on its own is usually too flawed, too narrow, or too easy to misread. But together, they start to tell a much more useful story. Our Lovable score has the following formula and weights: 35% NPS + 25% PMF + 20% CSAT + 20% CES Your mix might be different. But I’d really push every team to define what “love” actually means for their product, then measure it. Wrote more about how we built it here: https://lnkd.in/emQr6TFS And big shout out to Passionfroot, my awesome blog sponsor this week, who you should work with to scale your creator-led growth! #growth #lovable

BREAKING! The FDA just released this draft guidance, titled Artificial Intelligence-Enabled Device Software Functions: Lifecycle Management and Marketing Submission Recommendations, that aims to provide industry and FDA staff with a Total Product Life Cycle (TPLC) approach for developing, validating, and maintaining AI-enabled medical devices. The guidance is important even in its draft stage in providing more detailed, AI-specific instructions on what regulators expect in marketing submissions; and how developers can control AI bias. What’s new in it? 1) It requests clear explanations of how and why AI is used within the device. 2) It requires sponsors to provide adequate instructions, warnings, and limitations so that users understand the model’s outputs and scope (e.g., whether further tests or clinical judgment are needed). 3) Encourages sponsors to follow standard risk-management procedures; and stresses that misunderstanding or incorrect interpretation of the AI’s output is a major risk factor. 4) Recommends analyzing performance across subgroups to detect potential AI bias (e.g., different performance in underrepresented demographics). 5) Recommends robust testing (e.g., sensitivity, specificity, AUC, PPV/NPV) on datasets that match the intended clinical conditions. 6) Recognizes that AI performance may drift (e.g., as clinical practice changes), therefore sponsors are advised to maintain ongoing monitoring, identify performance deterioration, and enact timely mitigations. 7) Discusses AI-specific security threats (e.g., data poisoning, model inversion/stealing, adversarial inputs) and encourages sponsors to adopt threat modeling and testing (fuzz testing, penetration testing). 8) And proposed for public-facing FDA summaries (e.g., 510(k) Summaries, De Novo decision summaries) to foster user trust and better understanding of the model’s capabilities and limits.

👩🦰 Persona Spectrum For Inclusive Design (Figma Kit) (https://lnkd.in/eGD38hs4), a wonderful little accessibility tool for designers to include permanent, temporary and situational contexts in design decisions. Open sources, with all illustrations and assets for presentations and print. By 🐝 Mahana Delacour. --- 🔶 1. Accessibility ≠ Compliance We should never rely on automated accessibility testing alone to “ensure” accessibility. Compliance means that a user can use your product, but it doesn’t mean that it’s a great user experience. Manual testing makes sure that your users actually can meet their goals in their own context. It often feels daunting to get started, but small first steps are a great beginning. First, gather people interested in accessibility. Document what research was done, where the gaps are. And then try to include 5–12 users with disabilities in a dedicated accessibility testing. One way to find participants is to reach out to local chapters, local training centers, non-profits and public communities of users with disabilities in your country. You might want to add extra $25–$50 depending on disability transportation. Once you have access to users, run a small accessibility initiative around key flows in your products. Tap into critical touch points and research them. Eventually extend to components, patterns, flows, service design. A good target is to incorporate inclusive sampling into all research projects — at least 15% of usability testers should have a permanent, temporary or situational disability. --- 🔹 2. Building Accessibility Research From Scratch If you’d like to get started, I highly recommend to check “How We’ve Built Accessibility Research at Booking.com” (https://lnkd.in/eq_3zSPJ), a fantastic case study by Maya Alvarado on how to build accessibility practices and inclusive design into UX research from scratch. Maya highlights the idea of extending Microsoft's Inclusive Design Toolkit (https://lnkd.in/eN5J7EkJ) to meet specific user needs of a product. It adds a different dimension to disability considerations which might be less abstract and much easier to relate for the entire organization. And as Maya noted, inclusive design is about building a door that can be opened by anyone and lets everyone in. Accessibility isn’t a checklist — it’s a practice that goes way beyond compliance. A practice that involves actual people with actual disabilities throughout all UX research activities. More resources in the comments ↓

I made Physical AutoResearch sound simple (conceptually), but it took a village to pull off and lots of design thinking into the robot /loopcraft. The hardest part is everything we need to setup *before* pressing Enter. We released ENPIRE yesterday if you missed it: https://lnkd.in/gXeZzMYb Project site: https://lnkd.in/g3t4qS8Y Here's a behind-the-scene tour: 1. Safety harness Letting 8 robots run unattended overnight means safety has to be more than a hint in the system prompt. ENPIRE hardwires it in 2 layers: (1) hard kinematic limit that trips an immediate task failure and auto-resets as soon as a robot leaves its safety envelope, and (2) a torque-limited compliant gripper so a bad contact or misaligned insertion ends in a safe stall, instead of crushing the robot or the object at hand. We make safety more conservative than usual so humans can sleep tight. In reality, we still need a few human operators to watch over the "robots of loving grace". 2. Definition of /done An agent that can edit its own reward will game it for sure. ENPIRE fixes the goalposts before the fleet can move them. Here's the recipe: Collect a few minutes of success & failure demos -> Ask agent to write code using computer vision tools to classify success and measure against groundtruth -> Agent hill-climbs on classifier until reliably good -> This classifier becomes the real-time reward function that directly computes on sensor streams -> *Freeze* the reward function before AutoResearch. It's sacred, enshrined in a Gym env that no one can touch. 3. System telemetry design Robot-seconds is by far the scarcest resource, followed by GPU-seconds, and finally tokens. We instrument all three and surface them to ENPIRE for live resource awareness rather than letting it hill-climb in a vacuum. We define: - Mean Robot Utilization ("MRU"): the fraction of wall-clock time when the robot is actively executing an experiment. Otherwise the hardware is sitting idle and waiting for the next code commit. - Mean Token Utilization ("MTU"): tokens consumed per minute, our proxy for how hard the agent is actually thinking. A low MTU means the agent is stalled, waiting on a robot rollout to finish instead of doing research. - GPU utilization: fraction of wall-clock time when GPU is active. ... and evaluate on two budget-to-outcome metrics: 1. Tokens-to-Success: token budget the fleet burns to complete /goal. 2. Time-to-Success: wall-clock time to /goal

Product-market fit (PMF) isn't a binary. The reality is that there are *shades* of PMF and you need an *action plan* to get there. Maja Voje — better known as the GTM Strategist — has worked with 350+ startups to help them achieve & expand on PMF. She's now sharing her tested frameworks with the rest of us. Here's the TL;DR: 1️⃣ Proof of concept: Get 10 testers - These tend to come from your personal network, advisors or warm outreach (from a founder) with a hook - Show "problem-solution fit" by starting to document "can we even solve this problem?" metrics with case studies 2️⃣ Proof of monetization: Get 5 paying customers - These come from retained PoC testers, cold outreach to adjacent segments, case studies sent as warm outreach with a hook, or via influential people in your network - Pro tip: you need an early customer profile before you can get to an ideal customer profile (ICP) 3️⃣ Proof of 1+ scalable GTM motion: Reach 20+ paying customers - Your GTM options: inbound (content), outbound (cold outreach), paid digital, community, partners, ABX and/or PLG - Pro tip: you need differentiated positioning to unlock this GTM motion; Maja's recommendation is to always position in relation to *something* (a service, DIY process, doing nothing or direct competitors) 4️⃣ Proof of a sustainable business model: Reach 50+ paying customers - If you were to only use this 1+ scalable GTM motion, would you be able to become break-even / profitable? - Look at: retention/churn, acquisition costs, customer referenceability 5️⃣ Proof of market expansion: Reach 100+ paying customers in 2+ markets - There's now clear evidence that you're ready to win on more fronts: opening new markets, launching new products, selling to new personas --- Read the full piece in Growth Unhinged: https://lnkd.in/guUFj-5H My favorite quote: "I like to think of PMF as a cycle... Every time I fail to validate something, I remember that Nokia started with toilet paper, Lamborghini with tractors, and McDonald’s with hot dogs." Can't wait to hear what you think 🙏 #pmf #startup #gtm

I run a £20K/month research process through Perplexity for free. Founders don't realise what they're sitting on: When I was building Searchable, I needed: - Deep market analysis on AI search adoption - Competitive intelligence on legacy SEO platforms - Regulatory landscape research on AI governance Hiring analysts would've cost £15K-£25K per project. Instead, I built a prompting system that turned Perplexity into a Harvard research team. Here's the system I use to get the most out of Perplexity: ✅ Time-bound everything → "Last 24 months only" filters recency bias ✅ Demand structure → Tables, headings, sections beat wall-of-text responses ✅ Force citations → "Cite all claims" makes output defensible ✅ Require both sides → Evidence for, against, and what's uncertain ✅ End with action → "So what?" or "What should someone do next?" ✅ Apply domain knowledge → Review sources, validate, add your expertise And the prompts to get started: (Find the full prompts in the sheet 👇) 1. Market Landscape Snapshot Breaks down market size, growth rate, main players, and 3-5 trends that matter.  Ends with practical "So what?" No speculation allowed. 2. Competitive Comparison Breakdown Creates a positioning table comparing offerings, pricing, and distribution.   Flags contradictions in public information. 3. Trend Validation Check Determines if a trend is real or hype by gathering evidence on both sides.  Verdict: hype, early signal, or established shift. 4. Deep Dive on a Single Question Goes beyond surface summaries.  Shows where experts agree, disagree, and what nuance is missed. 5. Buyer & User Insight Synthesis Analyses real customer language from reviews, forums, and discussions.  Surfaces pain points and desired outcomes. 6. Regulation & Risk Overview Maps the regulatory landscape by region.  Focuses on what affects real decisions, not theoretical compliance theatre. 7. Evidence-Based Opinion Builder Lays out the strongest arguments on both sides.  Highlights where evidence is strong versus mixed. 8. Research-to-Decision Summary Structures output as: what's known, key risks and sensible next steps.  Practical decision framework, not predictions. Market validation that used to take days now takes minutes. And the output quality isn't just faster. It's often better because it's pulling from all sources simultaneously. Have you tried Perplexity for research?  Leave a comment below with your thoughts.  If you want your business to show up in AI tools like ChatGPT and Perplexity,  Searchable is the simplest way to have it done for you. It's an autonomous SEO & AEO Growth Engineer that analyses, fixes, and scales your website to drive customers. Learn more and start your 14-day free trial:  https://lnkd.in/epgXyFmi ♻️ Repost to help your network to level up their research.  And follow Chris Donnelly for more on building with AI.  

At Amazon, we would often spend months working on a single paragraph of the PR/FAQ for a new product idea. This was the "problem paragraph". Done well, it could lead to a successful product. Done wrong, it will lead to failure. Here is how to write a successful problem paragraph: The “problem paragraph” defines the customer problem you’re solving. Without this, you will build a product that doesn’t address a customer pain point. It shows whether you truly understand your customer's needs, not just your company’s capabilities. To write this paragraph, start by precisely identifying the customer segment that will be served by your product. Great products are built for specific people with specific needs. For instance, designing a car for single urban professionals under 35 differs significantly from designing for suburban families with three kids and a dog. If you think your product is for everyone, you’re mistaken. A strong way to begin your paragraph is: “Today, [customer segment] has [problem], which they currently solve using [methods A, B, and C]…” Next, quantify the problem: → How large is the segment? (e.g., 17 million households) → What methods do they use? (e.g., 45% use A, 25% use B, 30% use C) → What are the tradeoffs? (e.g., speed, cost, quality) Here’s an example for a hypothetical robot vacuum product: “Today, 15 million busy urban and suburban professionals earning between $100,000 and $200,000 struggle to find the time and energy to keep their homes clean. Approximately 30% of these households use traditional vacuuming, which requires up to 2 hours per week. 55% hire a cleaner at a minimum of $50/week, and 15% use robot vacuums that cost $600 plus $100/year in maintenance, while leaving behind up to 30% of dust and dirt.” This problem paragraph quantifies the customer problem in terms of money, time, and other metrics where possible (in this case, the dust and dirt left behind). The problem should always be quantified; otherwise, how can you assess the potential value of a product that solves it? Well-defined customer problems are built on data-based insights. Insights are gleaned from swimming in data and metrics. This includes customer usage metrics, process or operations metrics, user interviews, demographic data, customer feedback, customer support data and anecdotes. The more data-based and specific your insight, the more accurate and helpful your problem paragraph will be. This is why the process can take months. However, distilling these quantified insights into a single paragraph gives you the best chance at building a truly useful product. At Amazon, this paragraph was always the most debated section in a PR/FAQ. This is because getting the problem wrong is the worst mistake you can make in building a product. Everywhere else, you can pivot. But if the problem is incorrectly diagnosed, nothing else matters. (cont. in comments)

I’m a tech buyer with $75K/yr budget for sales tools. I won’t approve any purchase without a business case. Yet 99% of AEs just send me 30-page decks I’ll never read. If you want to close me, skip the fluff and send me the numbers. Here are the 4 pillars of a bullet-proof business case: 1. Current Challenges Describe the pain in MY language, not your pitch deck’s. Show you understand my environment, my KPIs, and my priorities. One to two sentences are enough if it’s sharp. 2. Cost of Inaction Put a number on what I lose by doing nothing; continued loss of revenue, wasted hours, missed opportunities. If you can’t quantify it, you’ll lose me here. 3. Time to Value Tell me exactly how quickly I’ll see results. Weeks beat months. Days beat weeks. Show me proof from similar customers in my size and space. Don’t show me how Oracle or Salesforce did it if I’m running a company at one-tenth their size and a few zeroes behind in revenue. 4. Predicted Objections Preempt the, “Yeah, but…” I’ll have in my head; budget, security, integration risk, adoption. I actually have a section in my business cases called:  “Gal Aga, you might be thinking, why can’t we just use the tools we have?” Because I know that’s always his first question. — This is just an overview. If you want to go deep, I have you covered. Writing a full business case can be PAINFUL. So Aligned built a free playbook to help: Access it here: https://lnkd.in/gZg_kCvr

Sometimes, best solutions are the simplest ones. When Colgate hit a plateau in growth, they faced a crucial challenge: How do we get people to use more toothpaste without changing the product itself? Most companies might have gone big — a flashy marketing blitz, a brand-new product range, or an expensive rebrand. Colgate, however, chose a far simpler and smarter route. They widened the opening of their toothpaste tube. No new formula. No fancy flavors. No dramatic packaging overhaul. Just a slightly bigger hole. This tiny tweak meant people unknowingly squeezed out more toothpaste each time. Tubes emptied faster, and sales went up — all without consumers even realizing why. It was a masterstroke in behavioral psychology: a change so subtle it flew under the radar, yet powerful enough to shift buying habits. A reminder that sometimes, the most effective growth moves are the quiet ones.