Peter Anderson

One of the best quants I've ever worked with ran a team of 4 that peaked $300M+/year in delta one MM. That's nearly 100x the profit per employee of NVDA and top AI startups. His stack? STL, ggplot, SGE. Whenever I leaned over his shoulder, he was writing some basic C++ and R. Fast forward many years, I get messages asking me why we don't use Apache #Iceberg, #DataFusion, or whatever it is. I get this often enough that I've coined a term for this: Apache-driven development (#ADD) That is: blindly piling on trendy Apache tools until the stack collapses under its own weight. Codebases with this antipattern are usually visited by its close relatives, like resume-driven development (RDD) and npm dependency hell. They often have performance bottlenecks in millions of lines of Java that they've never looked over. I know which kind of stack I prefer to maintain.

1,049

58 Comments

I can see the future of autonomous taxis as they would certainly help with peak times demand and other use cases, but I can also see how there will be many challenges of making it really work at scale. Here is one, a very ''analogue'' problem, which I am sure leaves Waymo engineers wake at night. Local fleet management operations are crucial, meaning, in every city where AV taxis are launching, the underlying infrastructure forms the backbone and ensures the fleet runs smoothly.

235

74 Comments

Our extraction agent in LlamaExtract has gotten a huge upgrade in capabilities 🔥- it can read a complex, multimodal document and extract into a Pydantic schema with 3+ layers of nesting.  A sneak peek on what’s possible: I fed in an equity research report which is completely tiled over with text, table scans, and charts, and extracted out a fully structured research summary with the following nested fields: 1. Report metadata - high-level company info like name, ticker, price 2. Company overview - describe business models, segments, geographic breakdown 3. Financial metrics - key ratios like EBITDA + margins/revenue growth over time 4. Investment thesis 5. Valuation 6. Risk factors Huge shoutout to Neeraj Pradhan for all the continued work on this. Sign up to LlamaExtract: https://lnkd.in/g9Wpqn7w Example LlamaIndex notebooks here: https://lnkd.in/gsvKqrNV

489

14 Comments

The SF/Bangalore corridor is finally real. I’ve spent the past few weeks across Stanford, YC, and in closed-room sessions with infra builders, researchers and operators in SF. Founders, capital and ideas are moving both ways. The physics on each side are totally different, and complimentary. San Francisco is running the cognition game: -Foundational research and frontier labs (Stanford University, University of California, Berkeley, OpenAI, Anthropic) -Dense enterprise networks and deep capital (trust and network driven) -The only place where new architectures, RL, security, compliance and eval systems truly compound. But it’s also brutally competitive, expensive, and narrative-driven. The Valley doesn't fund tourism, it funds ambition and originality. To win here, you need presence, references, domain fluency, enterprise relationships, and patience for immigration and burn. India is running the behaviour game: -Billion-user distribution, sovereign + vernacular data, and multimodal behaviour by default -Hybrid human-in-loop workflows and the cheapest feedback cycles on the planet. - Smart hardware, robotics, defence, and space (cost curves that actually compound, government support, talent density) India doesn’t trade on story; it trades on usage, retention, and trust. India’s advantages are structural: engineering ambition + hunger, cultural comfort with hybrid workflows, and an economy that rewards products people actually use. For decades, the flight path was one-way: talent left, capital came back. Now it’s a loop. Founders still fly west for Y Combinator, networks, and research; American companies and diaspora founders are flying east for the scale, cost efficiency and billion-user testbeds. SF still sometimes looks down on India. And India still gets performative about SF. But credibility is earned, not borrowed. Geography amplifies your work, it doesn’t replace it. You can now build in both worlds. Research where depth lives. Distribute where behaviour compounds. The corridor is now operational, not aspirational.

612

74 Comments

This section on how dbt Labs transitioned from a purely PLG motion to layering on enterprise is a fascinating one. Two instructive passages that I have bolded (below from First Round Review's path to PMF series featuring dbt Labs) TLDR: GTM comprises your ICP, channel, and message. When you transition from PLG / bottom up to Enterprise / top down motion, naturally your ICP and channel also changes, but your messaging / proposition needs to change too, e.g., the enterprise may be multiple personas and are buying assurance as much as solution. --- "Handy found product-market fit organically for dbt as an open-source tool mostly used by data practitioners and developers. But a few years into running a commercial business, he realized he had to build a growth curve all over again with C-suite data leaders. “Even though we had an unbelievable amount of market pull, as we initially commercialized, it wasn’t easy for us to transform this open-source command line tool into a product that enterprises would pay a million dollars for,” says Handy. “When you have enough product-market fit, sometimes it allows you to get away with not being super tight on product marketing or sales motions. So around 2022, we went from this gigantic acceleration curve and overnight we realized, we have to sit at the adults’ table and figure things out real fast,” says Handy. After the PLG flame started to fizzle, Handy turned his attention to layering on a sales-led motion for the cloud platform. “We had to focus our efforts on telling cohesive stories to senior data leaders. We had to have a very clear, explainable answer to the question, ‘Why should I use the commercial product and not the open-source product? And it had to be digestible by someone with a C in their title,” he says. Handy’s answer: dbt Cloud can handle complex data for companies of every size. “The longer people used dbt, the more complex their code became,” he says. “It was a problem for the most sophisticated dbt users, who were often at the largest companies. So there was a real opportunity for us to step in and solve that for them with dbt Cloud.” To tell that story to enterprise customers, Handy relied on data, naturally. “At a user conference we presented a chart that showed the number of dbt projects that had a certain number of models in them — over 100, over 1,000, et cetera,” he says. “We watched that number climb and we knew as users ourselves, ‘Oh my God, trying to work in a dbt project with 5,000 models in it is challenging.’ So we started with that quantitative data point and asked folks in our community about their experiences with these very large, complex dbt projects, and validated that this was a pain in the ass without a cloud platform.”

23

4 Comments

We got LLM query latency down to 33ms. Constant. Regardless of context size. Streaming inference is broken. Every framework today (vLLM, TensorRT-LLM, SGLang, llama.cpp) processes the full context when a query arrives. Context grows, latency grows. O(n). We built a new computational model for streaming data and achieved O(1) query latency. Results on streaming benchmarks with Llama-3.1-8B: - 33ms average query latency (vs 220-260ms for vLLM/TensorRT-LLM/llama.cpp/SGLang) - 7-8x faster than state-of-the-art inference engines - 28-120x faster than cloud APIs - Full quadratic attention preserved. No sparse approximations. No linear attention tradeoffs. We have a technical whitepaper covering the architecture, the math, and the systems work that made it possible. Looking for people to poke holes in it, discuss applications, or explore what else this enables. If you work on inference infrastructure or attention mechanisms, I would love to talk. [email protected] or DM. LayerScale, Inc.

711

87 Comments

Great to see how a computational physics research lab is using Pulse's API to extract LaTeX equations and mathematical notation from their 40+ year archive of papers. Their team built a specialized pipeline that preserves the semantic structure between equations, preserving relationships between variables, theorems, and proofs. What's remarkable is how Pulse correctly identifies and extracts nested mathematical structures (matrices within summations within integrals) while maintaining proper symbol relationships. Our mission is to turn previously inaccessible computational knowledge into structured, searchable data that accelerates novel research. If you're a research team interested in trialing out Pulse, send me a DM for some trial credits! Sample LaTeX output attached below.

64

1 Comment