Seth Paydar

If a human can accidentally do it, so can Claude. - Restrict Claude's access to Terraform commands - Restrict Claude's access to credentials - Add delete protections at the cloud provider level or database level - Store backups in a separate environment

2

1 Comment

You can now run 70B LLMs on a 4GB GPU. That is not a typo. AirLLM just made massive models usable on low-memory hardware. (And it even runs 405B Llama 3.1 on 8GB VRAM) Why this matters:   - No need for expensive cloud GPUs   - Large models now work on laptops and small machines   - Local experimentation just got much cheaper  This quietly changes who gets to work with frontier-scale models. Link in the first comment. #AI #LLMs #OpenSource #LocalAI #GenerativeAI #DeveloperTools #MachineLearning

63

8 Comments

I’ve noticed that observability is often an afterthought (at best) when developers build LLM-powered applications. That bothers me. A lot. When we’re working with LLMs, observability should be a first-class citizen, baked into the design from day one. Not something we scramble to add once things go wrong in production. On Thursday, 16th October, I’ll be presenting at the Brisbane AWS User Group on this topic, discussing the what, why, and how of LLM observability. Link is in the comments. We’ll look at how emerging frameworks like AgentCore and Strands SDK are tackling observability, and how they compare to other tools like W&B and Opik. If you’re building, shipping, or scaling AI-powered applications, I’d love for you to join the conversation. Please ask questions! #AWS #LLMOps #Observability #AI #Serverless

20

3 Comments

I just published Vol. 109 of "Top Information Retrieval Papers of the Week" on Substack. My Substack newsletter features the 7-10 most notable research papers on information retrieval (including recommender systems, search & ranking, etc.) from each week, with a brief summary, and links to the paper/codebase. This week’s newsletter highlights the following research work: 📚 Zero-Shot Enhancement of LLM Rerankers through BM25 Score Integration, from Seetharaman et al. 📚 A Visual Analytics Framework for Debugging Graph-based RAG Systems, from Zhejiang University 📚 Production-Scale Generative Recommendations with Reinforcement Learning, from Kuaishou 📚 A Generative Approach to Cold-Start Recommendation via Sequential User Modeling, from ByteDance 📚 A Reference-Anchored Approach for Zero-Shot Document Ranking, from Li et al. 📚 Structure-Aware Code Chunking for Enhanced Retrieval-Augmented Generation, from CMU 📚 A Modular Open-Source Framework for Efficient Retrieval-Augmented Generation Research, from Zhang et al. 📚 A Learning-to-Rank Framework for Dynamic Retriever Selection in RAG Systems, from CMU 📚 A Comprehensive Review of Multi-Interest Recommendation, from Li et al. 📚 Hard Negative Mining through Hierarchical User Grouping in Recommendation Systems, from Credit Karma #InformationRetrieval #ResearchPapers #CuratedContent #Newsletter  #substack

46

2 Comments