Mostafa Majidpour

Another year of rapid AI advances has created more opportunities than ever for anyone — including those just entering the field — to build software. In fact, many companies just can’t find enough skilled AI talent. Every winter holiday, I spend some time learning and building, and I hope you will too. This helps me sharpen old skills and learn new ones, and it can help you grow your career in tech. To be skilled at building AI systems, I recommend that you: - Take AI courses - Practice building AI systems - (Optionally) read research papers Let me share why each of these is important. I’ve heard some developers advise others to just plunge into building things without worrying about learning. This is bad advice! Unless you’re already surrounded by a community of experienced AI developers, plunging into building without understanding the foundations of AI means you’ll risk reinventing the wheel or — more likely — reinventing the wheel badly! For example, during interviews with job candidates, I have spoken with developers who reinvented standard RAG document chunking strategies, duplicated existing evaluation techniques for Agentic AI, or ended up with messy LLM context management code. If they had taken a couple of relevant courses, they would have better understood the building blocks that already exist. They could still rebuild these blocks from scratch if they wished, or perhaps even invent something superior to existing solutions, but they would have avoided weeks of unnecessary work. So structured learning is important! Moreover, I find taking courses really fun. Rather than watching Netflix, I prefer watching a course by a knowledgeable AI instructor any day! At the same time, taking courses alone isn’t enough. There are many lessons that you’ll gain only from hands-on practice. Learning the theory behind how an airplane works is very important to becoming a pilot, but no one has ever learned to be a pilot just by taking courses. At some point, jumping into the pilot's seat is critical! The good news is that by learning to use highly agentic coders, the process of building is the easiest it has ever been. And learning about AI building blocks might inspire you with new ideas for things to build. If I’m not feeling inspired about what projects to work on, I will usually either take courses or read research papers, and after doing this for a while, I always end up with many new ideas. Moreover, I find building really fun, and I hope you will too! [Truncated for length. Full text: https://lnkd.in/gwv8nsgN ]

3,939

259 Comments

New short course: DSPy: Build and Optimize Agentic Apps DSPy is a powerful open-source framework for automatically tuning prompts for GenAI applications. In this course, you'll learn to use DSPy, together with MLflow. This is built in partnership with Databricks and taught by Chen Qian, co-lead of the DSPy framework. Many AI builders spend hours hand-tuning prompts. When given a set of evals, DSPy automates this process. It’s especially useful for optimizing prompts, including few-shot prompts, in complex agentic AI workflows. Further, if you switch an application to a newer LLM, performance can degrade if your prompts were optimized to the previous model. DSPy automatically optimizes the entire system for the new LLM as well, using just a few evaluation examples. This course teaches DSPy works, and best practices for using it. You’ll write programs using DSPy’s signature-based programming model, debug them with MLflow tracing -- to gain visibility into how different parts of a pipeline, as well as how the overall system, are performing -- and automatically improve their accuracy with DSPy Optimizer. Please sign up here: https://lnkd.in/gdjae8AX

4,672

190 Comments

Around 9 minutes in, Kevin Roose and Casey Newton make up a fanciful story about how AI is organized at Meta, why Meta has an open source strategy. They got it completely wrong. A big mistake is to confuse research (particularly fundamental research) and product development. These are two very different activities. FAIR does research. I do research. Product development is done by a separate organization called GenAI. GenAI produces Llama and Meta AI, not FAIR, even if the original Llama came from FAIR. First, they say that I run AI R&D at Meta. That's false. I was running FAIR, i.e. AI *research* (not development) from 2014 to 2018. In 2018, I became Chief AI Scientist, meaning I became an Individual Contributor, with no organization to manage. I like that, because that allows me to focus on research. Since 2018, FAIR has been run by Antoine Bordes, Joelle Pineau, and now Rob Fergus. Second, they say that AR/VR and feed and ad ranking distracted us from the more ambitious goal of building truly intelligent machines. They say that it distracted us from working on LLM. They also say that I don't believe in LLMs, which is why we are (allegedly) behind now. That's ridiculous. in 2018, Meta created a whole AI organization called Meta IA led by Jerome Pesenti, overseeing both FAIR (fundamental AI research) and FAIAR (applied AI research). FAIAR was the organization focusing on practical applications of AI deployed in products, including ad and feed ranking, content moderation, integrity, translation, etc. FAIR was very much working on chatbots, LLMs and transformers. In fact, many of the techniques that have now become common place were invented at FAIR, a number of them as part of the ParlAI project led by Jason Weston (which can be found here https://lnkd.in/e5XSD8JD ). Late 2018, Google published the BERT model, a large encoder-decoder transformer system trained in a self-supervised manner. Inspired by this work, a few months later, FAIR open-sourced RoBERTa, which became the workhorse of many NLP research groups around the world ( https://lnkd.in/ejirXRQW ) NLP research soon shifted to "decoder only" architectures, pioneered by OpenAI's GPT2. In 2022, FAIR open-sourced OPT-175b, which was Meta's open-source response to OpenAI's closed-source GPT3 ( https://lnkd.in/eWjp-qUx ). Meanwhile, Meta AI was dissolved: AI technology had become so important to products that the FAIAR development groups were moved to the product groups that used their technology. FAIR was moved under the Reality Labs - Research organization. ..... https://lnkd.in/eudTeyDd

1,808

105 Comments

An interview with Jeff Jarvis and Jason Howell for the AI Inside podcast. We discuss the limitations of current large language models, why human-level AI is still years away, Meta's open-source approach with LLAMA, and the vision for AI assistants that understand the physical world. https://lnkd.in/e8-umf4m

300

24 Comments

No pretraining. No datasets. Just pure inference-time gradient descent on the target ARC-AGI puzzle itself, solving 20% of the evaluation set. Introducing *ARC‑AGI Without Pretraining* Check out MLD PhD student Isaac Liao's technical blog post here: https://lnkd.in/eY_kvbQj

17

Tuning the step size for gradient-based samplers is still one of the biggest practical headaches in Bayesian computation. For Metropolis-Hastings-based algorithms like the random-walk, MALA, HMC, etc., you can tune the step size based on an optimal acceptance rate. But, in machine learning, it's more common to use non–Metropolis–Hastings algorithms like ULA, SGLD, SVGD, etc. Here, a “good” step size is crucial: Too small → painfully slow mixing and wasted compute Too large → numerical instability and bias And in high dimensions, the usual rules of thumb break down quickly In a new paper led by Louis Sharrock, we tackle this problem by viewing sampling as an optimisation problem on the space of probability measures and designing a tuning-free step size schedule directly from a functional upper bound. E.g., a non-asymptotic bound on the Kullback-Leibler divergence. The result is FUSE: an adaptive step size rule that you can plug into ULA, SGLD, SVGD and other related Wasserstein gradient-flow methods to eliminate manual step size tuning, while retaining (almost) the same guarantees as if you had chosen the optimal constant step size in hindsight. We've written a blog post that explains the key ideas, with figures and intuition: 👉 Blog: https://lnkd.in/ee9K9biX And the full technical details are in the paper: 📄 Sharrock & Nemeth (2025), Tuning-Free Sampling via Optimization on the Space of Probability Measures https://lnkd.in/enwch4NE If you work with Langevin-based samplers, SGLD for large-scale Bayesian inference, or particle methods like SVGD and are tired of endlessly fiddling with learning rates, you might find FUSE a useful perspective — and, hopefully, a practical solution. #BayesianInference #MonteCarlo #MCMC #ProbabilisticAI #Optimization #MachineLearningResearch

105

2 Comments

In collaborative machine learning (CML), early contributors take on more risk and encourage participation from wait-and-see parties, but existing CML frameworks treat every party as if they join simultaneously. The #NeurIPS2025 paper of Jiangwei Chen, Nguyen Pham, Rachael Sim, Arun Verma, Zhaoxuan Wu, Chuan Sheng Foo, and Bryan Kian Hsiang Low proposes a time-aware CML framework that rewards parties not just for what they contribute, but when they contribute: •⁠ ⁠8 incentives extending fairness with time-aware notions that incentivize early participation while preventing low-quality early submissions. •⁠ ⁠2 reward distribution methods that theoretically satisfy all incentives and recover the #ShapleyValue in time-agnostic cases. •⁠ ⁠A practical recipe for constructing valuation functions and reward realization, with empirical validation on synthetic and real-world data. Find our more at our posters! #NeurIPS2025 Thu, Dec 4, Exhibit Hall C, D, E #1104 #EurIPS2025 Wed, Dec 3, Poster Stand #56 Paper: https://lnkd.in/g_yM6j39 #FederatedLearning

50

Mixed curvature models have had a lot of success in graph neural network training, where local embeddings account for local curvature. This paper applies mixed curvature techniques to knowledge graph embeddings and Tucker decompositions (a favorite tool from psychometrics) to fill in knowledge gaps in knowledge graphs. https://lnkd.in/euM9z839

65

5 Comments

📝 Announcing our paper that proposes a unified cognitive and computational framework for Artificial General Intelligence (AGI) -- going beyond token-level predictions -- one that emphasizes modular reasoning, memory, agentic behavior, and ethical alignment 🔹 𝐓𝐡𝐢𝐧𝐤𝐢𝐧𝐠 𝐁𝐞𝐲𝐨𝐧𝐝 𝐓𝐨𝐤𝐞𝐧𝐬: 𝐅𝐫𝐨𝐦 𝐁𝐫𝐚𝐢𝐧‑𝐈𝐧𝐬𝐩𝐢𝐫𝐞𝐝 𝐈𝐧𝐭𝐞𝐥𝐥𝐢𝐠𝐞𝐧𝐜𝐞 𝐭𝐨 𝐂𝐨𝐠𝐧𝐢𝐭𝐢𝐯𝐞 𝐅𝐨𝐮𝐧𝐝𝐚𝐭𝐢𝐨𝐧𝐬 𝐟𝐨𝐫 𝐀𝐫𝐭𝐢𝐟𝐢𝐜𝐢𝐚𝐥 𝐆𝐞𝐧𝐞𝐫𝐚𝐥 𝐈𝐧𝐭𝐞𝐥𝐥𝐢𝐠𝐞𝐧𝐜𝐞 𝐚𝐧𝐝 𝐢𝐭𝐬 𝐒𝐨𝐜𝐢𝐞𝐭𝐚𝐥 𝐈𝐦𝐩𝐚𝐜𝐭 🔹 In collaboration with University of Central Florida, Cornell University, UT MD Anderson Cancer Center UTHealth Houston Graduate School of Biomedical Sciences, Toronto Metropolitan University, University of Oxford, Torrens University Australia, Obuda University, Amazon others. 🔹 Paper: https://lnkd.in/gqKUV4Mr ✍🏼 Authors: Rizwan Qureshi, Ranjan Sapkota, Abbas Shah, Amgad Muneer, Anas Zafar, Ashmal Vayani, Maged Shoman, PhD, Abdelrahman Eldaly, Kai Zhang, Ferhat Sadak, Shaina Raza, PhD, Xinqi Fan, Ravid Shwartz Ziv, Hong Yang, Vinija Jain, Aman Chadha, Manoj Karkee, @Jia Wu, Philip Torr, FREng, FRS, Seyedali Mirjalili ➡️ 𝐊𝐞𝐲 𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬 𝐨𝐟 𝐓𝐡𝐢𝐧𝐤𝐢𝐧𝐠 𝐁𝐞𝐲𝐨𝐧𝐝 𝐓𝐨𝐤𝐞𝐧𝐬' 𝐂𝐨𝐠𝐧𝐢𝐭𝐢𝐯𝐞‑𝐂𝐨𝐦𝐩𝐮𝐭𝐚𝐭𝐢𝐨𝐧𝐚𝐥 𝐀𝐆𝐈 𝐅𝐫𝐚𝐦𝐞𝐰𝐨𝐫𝐤: 🧠 𝐅𝐨𝐮𝐧𝐝𝐚𝐭𝐢𝐨𝐧𝐚𝐥 𝐅𝐫𝐚𝐦𝐞𝐰𝐨𝐫𝐤: Integrates cognitive neuroscience, psychology, and AI to define AGI via modular reasoning, persistent memory, agentic behavior, vision-language grounding, and embodied interaction. 🔗 𝐁𝐞𝐲𝐨𝐧𝐝 𝐓𝐨𝐤𝐞𝐧‑𝐏𝐫𝐞𝐝𝐢𝐜𝐭𝐢𝐨𝐧: Critiques token-level models like GPT-4.5 and Claude 3.5, advocating for test-time adaptation, dynamic planning, and training-free grounding through retrieval-augmented agentic systems. 🚀 𝐑𝐨𝐚𝐝𝐦𝐚𝐩 𝐚𝐧𝐝 𝐂𝐨𝐧𝐭𝐫𝐢𝐛𝐮𝐭𝐢𝐨𝐧𝐬: Proposes a roadmap for AGI through neuro-symbolic learning, value alignment, multimodal cognition, and cognitive scaffolding for transparent, socially integrated systems.

221

9 Comments

Fine-Tuning LLMs — The Smart Way Forward Fine-tuning large language models (LLMs) used to mean retraining billions of parameters, demanding insane compute and memory. For most teams, that’s simply not feasible. But we’ve come a long way. Today, we can fine-tune LLMs efficiently using parameter-efficient fine-tuning (PEFT) techniques. Here are five popular approaches redefining what’s possible: 🔹 LoRA: Train small low-rank matrices (A & B) instead of full weights. 🔹 LoRA-FA: Freeze matrix A to reduce memory needs, train only B. 🔹 VeRA: Use shared, random A & B across layers; learn small layer-specific scalars. 🔹 Delta-LoRA: Apply the difference between A·B matrices to the main weights W. 🔹 LoRA+: Boost learning rate selectively for B to optimize convergence. These approaches make it practical and scalable to specialize models for healthcare, legal, customer support, or even your internal business processes — without breaking the bank. Whether you’re building an internal AI assistant or a vertical-specific GenAI tool, PEFT is the future. Read more: https://lnkd.in/gt4J4j2Y #LLM #FineTuning #AI #MachineLearning #LangChain #GenAI #LoRA #PEFT #AIProducts #ParameterEfficientTuning #OpenSourceAI

10

2 Comments