Issue #662
July 23, 2026

Hello!

Once a week, we write this email to share the links we thought were worth sharing in the Data Science, ML, AI, Data Visualization, and ML/Data Engineering worlds.

And now…let’s dive into some interesting links from this week.

Editor's Picks

• How to make a mesh dance in real time to music
  Audio transforms into a frequency space, where filtering becomes easy. The nice surprise is that meshes have a frequency space too…Take a drum head. Hit it and it doesn’t wobble arbitrarily; it rings in a discrete set of standing waves whose shapes are determined entirely by the drum’s geometry. Mark Kac asked in 1966 whether you could run that backwards: can one hear the shape of a drum?…On a triangle mesh those standing waves are the eigenvectors of the discrete Laplacian, its manifold harmonics. Low eigenvalues are slow global wobbles, high eigenvalues fine ripples: the same low-to-high ordering an FFT gives you, over a surface instead of over time…With this you can let the song drive the shape and get Kac’s question the other way. instead of hearing a shape, you see a sound…
  

• Random Selection and Democracy in Athens and Asimov
  Let’s look at some aspects of random selection as it relates to democracy through these examples. This post contains discussion questions and activities that could be used for a high school or undergraduate class discussion about the “fairness” of simple random, stratified, and systematic sampling — and of nonrandom selection methods. Learning objectives include deepening understanding of these sampling methods, distinguishing between proportional and disproportional allocation, calculating sampling weights for a stratified sample, and relating advantages and disadvantages of methods for political candidate selection to statistical principles of randomization. The footnotes contain some questions for an advanced undergraduate or graduate-level sampling class…

• London’s most equidistant pub
  London’s most equidistant pub is The Greene Man at 383 Euston Road, on the corner of Great Portland Street. Starting from where each of the 33 boroughs’ residents actually live, and arriving by 7pm on an ordinary September Thursday by public transport, journeys to it span 47 minutes between the nearest borough and the furthest, and no borough needs more than 70. No pub in London does better on both measures….The bigger finding is what nobody manages: not one of London’s 3,170 pubs is within an hour of all 33 boroughs….

.

What’s on your mind

This Week’s Poll:

.

Last Week’s Poll:

.

Data Science Articles & Videos

• You Could Have Come Up With Kimi Delta Attention
  In this post we are going to walk through the DeltaNet family of linear attention variants, two of which are used by the latest Qwen and Kimi model families, and show how you might have arrived at the same equations by asserting simple things about your hidden state. That is the route we will take: softmax attention  →  linear attention  →  DeltaNet  →  Gated DeltaNet  →  KDA Only after deriving KDA will we turn to the recurrent and chunkwise Triton programs that execute it…
  

• Anyone else measure team health by how long it takes a non-technical person to answer a basic data question? [Reddit]

  had a client once where the ops manager could just... answer things. revenue by region, churn this month, whatever. no ticket, no slack message to the data team, just opened metabase and pulled it herself i’ve been thinking about that a lot lately because most places i go into are nothing like that. analyst gets pinged for stuff that really shouldn’t need an analyst…

• Binary vector embeddings are so cool
  Vector embeddings by themselves are pretty neat. Binary quantized vector embeddings are extra impressive. In short, they can retain 95+% retrieval accuracy with 32x compression and ~25x retrieval speedup. Let’s get into how this works and why it’s so crazy…
  

• Some useful equations for biological processes
  Because mathematical modeling plays such a central role in biology and many other scientific disciplines, biologists need to be familiar with the most important mathematical functions. More importantly, they need to be able to “read” these functions and use their parameters to describe, interpret, and quantify biological processes. With this aim in mind, I have compiled a collection of the mathematical functions most commonly encountered in biology, explaining the meaning of their parameters, with particular emphasis on their biological interpretation rather than their mathematical properties…
  

• A collection of self-starters for nonlinear regression in R

  Usually, the first step in every nonlinear regression analysis is to select the function that best describes the phenomenon under study. The next step is to fit this function to the observed data, possibly by using some sort of nonlinear least squares algorithm…for the second step, the main problem is that nonlinear least squares algorithms are iterative, in the sense that they start from some initial guesses for the model parameters, which are continuously improved until the least squares solution is approximately reached. Quite often, providing such initial guesses for all model parameters becomes a problem: if our guesses are not close enough to the least squares estimates, the algorithm may stall and fail to converge. Or, even worse, it may converge to the wrong solution. How do we obtain good initial guesses for the model parameters? This is not easily accomplished, especially for students and practitioners. This is where self-starters come in handy…
  

• Prefer STRICT tables in SQLite
  In short: I prefer strict tables in SQLite because they avoid some datatype problems, such as putting text in number columns. SQLite has a feature that I think is underrated: strict tables. Strict tables help enforce rigid typing, preventing mistakes like putting text into integer columns. I like them, and wrote this post to promote their use!…


• Why did data scientists choose Rows to be observations? [Reddit]
  I’m a math guy so i don’t know sh*t about stats but in linalg we’re learning about covariance matrices, PCA, and SVD. In math we prefer columns to be observations because… well tradition, notational easy (Ax=b instead of xA=b if we used row vectors) and cuz we care abt linear transformations etc and its easier to think about columns. Idk. Mostly tradition tho. But then why did data scientists break the mold? What benefit did row observations possibly serve that Col vectors/observations couldn’t give?? I am jumping back and forth between notation and conventions and it’s hard to keep up…
  

• Cross-stitch prep for math nerds
  I’ve done quite a bit of fibre arts over the years, and surprisingly enough, picking up the right supplies in the right quantities is probably the hardest part. This post details how to figure out exactly what supplies you need via mathematics and geometry…
  

• Tech note: making your own V-I plots at home…Or, how to lose money with friends

  When working on my latest book, The Secret Life of Circuits, I wanted to keep the artwork real. My beef with the diagrams in popular electronics textbooks and online tutorials is that most of them are fake. At best, they’re retraced from ancient texts; at worst, they’re sketched from memory and can be charitably described as “inspired by true events”...
  

• Just brute force your embeddings
  I work with a lot of teams that don’t need the complexity of a vector database. They have ~1m documents to search. They have low query traffic, and write their embeddings all up front. They don’t need to buy a multi-million dollar vector database, or spend 6 months learning to operate it. For low enough n, just brute force the embeddings until you can’t bear to. As fellow search traveler Jo Kristian Bergum says “an exhaustive search may be all you need”…
  

• Going Bayesian automates your manual data analysis

  People usually talk about Bayesian modeling in terms of richer uncertainty estimates, prior knowledge, and posterior distributions. All true, and all worth the switch on their own. But there’s a benefit almost nobody talks about. Going Bayesian automates away the manual data analysis you’d otherwise do by hand. And once you experience it, you wonder how you ever worked any other way. Let me show you what I mean…
  

• How do you decide whether a data science problem really needs machine learning? [Reddit]
  In your experience, what factors help you decide between using a simple analytical approach and building a machine learning model? I’d love to hear the reasoning behind your decision-making process…
  

• Recursive Filters: SMA, EMA, Low‑Pass, and a Tiny Kalman
  While exploring optimizers I fell down the rabbit hole of recursive filters. This post is a compact, practical tour of smoothers you can use when measurements are noisy but latency and compute are tight. We’ll keep the math minimal, the intuition high, and focus on when and why to use SMA, EMA/low‑pass, and a tiny 1D Kalman…

Last Week's Newsletter's 3 Most Clicked Links

• What Do Today’s Data Science Graduates Commonly Lack? [Reddit]

• The Unreasonable Difficulty of Time Series Forecasting

• Why non-invasive glucose monitoring is hard

.
* Based on unique clicks.
** You can find last week's issue #661 here.

Cutting Room Floor

• The Unifying Capabilities of Cumulative Probability Semiparametric Models

• Colorblind II: Microscopy images and colour blindness

• Learning a few things about running SQLite

• RoboVista - Evaluating Vision-Language Models for Diverse Robot Applications

.

Thank you for joining us this week! :)

Stay Data Science-y!

All our best,
Hannah & Sebastian

Issue #661
July 23, 2026

Hello!

Once a week, we write this email to share the links we thought were worth sharing in the Data Science, ML, AI, Data Visualization, and ML/Data Engineering worlds.

And now…let’s dive into some interesting links from this week.

Editor's Picks

• Why non-invasive glucose monitoring is hard
  Continuous glucose monitoring on Apple Watch and other smartwatches has been “5-7 years away” for roughly a decade…I trained one of the first deep neural networks to detect signs of diabetes with consumer health sensors. This post will explain what makes glucose sensing so hard, what hardware and machine learning techniques have been tried, and try to describe which research techniques are actually feasible on a consumer device like an Apple Watch, Pixel, Oura, or Samsung Watch. Let’s start by explaining why an already-launched feature, blood oxygen sensing, actually works in practice using relatively cheap optical sensors…
  

• Navigating Challenges in Spatial Machine Learning
  Spatial machine learning has become a standard tool for producing environmental and geographic prediction maps. It is now relatively (technically) easy to combine field observations with remote sensing, climate, terrain, or other predictor layers and fit a strong machine learning model. The harder question is whether the resulting map is reliable, transferable, and reproducible…Spatial dependence, clustered and biased sampling, heterogeneous landscapes, and domain transfer all affect how models should be evaluated and interpreted. A model can appear accurate under a standard validation approach and still be unreliable where predictions are needed…

• Exploring Gymflation with AI
  You might be familiar with super hero inflation? Over time, super hero physiques on screen have become increasingly exaggerated. Batman’s progression from Adam West to Ben Affleck is a great example of this…Gymflation is much the same idea. As gym culture has skyrocketed, it seems like so too have people’s feats of strength - as recorded on social media. Going on YouTube or Instagram one gets the feeling that a 200kg deadlift is really rather average nowadays. But is it really true? Or are we just feeling the effects of the Algorithm, pushing extreme examples in our blue-lit faces? What follows is a little project to find out…

.

What’s on your mind

This Week’s Poll:

.

Last Week’s Poll:

.

Data Science Articles & Videos

• Easy A’s, Less Pay: The Long-Term Effects of Grade Inflation
  Average grades continue to rise in the United States, raising the question of how grade inflation impacts students. We provide comprehensive evidence on how teacher grading practices affect students' long-run success. Using administrative high school data from Los Angeles and from Maryland that is linked to postsecondary and earnings records, we develop and validate two teacher-level measures of grade inflation: one measuring average grade inflation and another measuring a teacher's propensity to give a passing grade…The cumulative impact is economically significant: a teacher with one standard deviation higher average grade inflation reduces the present discounted value of lifetime earnings of their students by $213,872 per year…
  

• What Do Today’s Data Science Graduates Commonly Lack? [Reddit]

  I often read comments from hiring managers and interviewers saying they’re disappointed with recent data science graduates. I’m curious, what do you think these graduates are lacking? If someone wants to become a data scientist, what skills should they focus on? Strong software engineering skills? Math and statistics? Something else?…

• How Hollywood Stopped Making Movies in Hollywood
  The economics behind where movies are filmed, why Hollywood left Los Angeles, and whether audiences can tell the difference…
  

• The Unreasonable Difficulty of Time Series Forecasting
  I’ve been thinking recently about what makes time series forecasting problems so difficult compared to other sequence learning tasks or IID Machine Learning problems…for many ML problems, we don’t have full information about the casual factors either yet are able to do something reasonable, or at least better than a random walk. Hence, this post investigates the nature of the forecasting problem and what makes it so much more difficult than classical machine learning…
  

• Holding the LLM Stack in Your Head

  A dependency-ordered walk through the modern LLM stack, from the linear algebra under a single attention head, through training and inference, out to agent protocols shipping in 2026. Ten arcs, eighty-odd posts. The goal isn't rigor, it's intuition that survives contact with real systems…
  

• How to Bake a [Golf Shot] Dispersion Pattern from Scratch
  I’ve built a realistic golf shot dispersion simulator and I’ve shared it for anyone to use…I wanted to write about variance in golf here, but when I was reviewing my theoretical, idealized dispersion patterns, I realized it was effectively pointless to start discussing variance in golf without generating realistic dispersion patterns. These golf shot dispersion patterns represent the inherent variance in the game, so we need the model to reflect them before looking at birdies and blow-ups…


• Colorblind: Checking figure accessibility for colour blind people
  When preparing images for publication, it is good practice to check how accessible they are for colour blind people. Using a simple bit of code, it is possible to check an image – or a whole figure – in ImageJ for accessibility. For example, Figure 1 from our recent paper. Originally looked like this:..
  

• Understanding the AI economy
  Google’s ATLAS is an expansive look at how people are using AI at work and in day-to-day life…Google is launching the first iteration of the AI & Economy ATLAS (Activity, Task, Landscape, and Adoption Study), an ongoing, large-scale, de-identified study of how people are using Google’s AI products and tools. ATLAS’s first dataset (v1.0) is built from 15 million aggregated and de-identified human-AI interactions across the Gemini App, AI Mode, and the Gemini API, which together are used by more than 1 billion people monthly. ATLAS v1.0 insights span more than 150 countries, 140 languages, 800 occupations, and 4,000 tasks; ATLAS is the most comprehensive look to date at how real people are using AI at scale…
  

• Handwriting but not typewriting leads to widespread brain connectivity: a high-density EEG study with implications for the classroom

  Our findings suggest that the spatiotemporal pattern from visual and proprioceptive information obtained through the precisely controlled hand movements when using a pen, contribute extensively to the brain’s connectivity patterns that promote learning…
  

• Guide to data tools landscape for developers
  Found yourself on a data project and have no idea what they all are talking about? Feel excluded from all the fun discussions in the office kitchen? If only there were a humongous guide going over all the concepts and buzzwords.….In this article, we will briefly go over the data lifecycle: where data comes from, how it’s handled, how it’s stored, and how it’s displayed. You will understand to which stage each particular tool belongs and which tasks it solves for people working with data. We won’t go into details about setting up each tool or comparing tools inside the same class in-depth. Trust me, the article will be pretty lengthy as is, even without going into any of those details…
  

• Scouting Uncertainty: adding value to data scouting results

  I have been looking at spreadsheets for football scouting for over a decade, and every now and then, my perception completely changes…Lately, I’ve been creating more and more on a meta level, mostly in data engineering. In that light, I wanted to share something I have introduced in my day-to-day data scouting when working with aggregated data: scouting uncertainty. The idea is to add an uncertainty score to the scouting score to add a level of trust towards the data. If the data is trustworthy and uncertainty is low, the quality of data scouting for that specific player will be higher…
  

• AI as an ETL and Report Builder? I’m tired. [Reddit]
  We have been developing a Data Platform (IaC, CI/CD, orchestration, data quality, governance, the works). Everything is already set-up except for the business logic. Quite understandable since we built everything from FOSS about 2 months ago and I’m the only data platform engineer/data engineer in the company. They aren’t also keen on spending money on managed solutions. Now, a director is pushing to scrap our project in favor of an AI as an ETL solution. Basically, use skills and AI to generate reports from source systems and have AI use python, pandas and SQL to generate reports. This AI as an ETL couldn’t get out of the demo phase because of data quality issues….
  

• Reading ‘Radical Uncertainty’
  I recently read John Kay and Mervyn King’s book Radical Uncertainty as part of an economics book club and I thought I would share some thoughts on the book here…Radical Uncertainty is about statistics, and especially about the problems with assuming that we can make probabilistic calculations while neglecting the possibility of things which we haven’t even contemplated happening. I was pretty fascinated by the first half of the book, but it does become quite long-winded in the second half, and repeats many of its points over and over again. The book makes a lot of good points, but I would also like to point out some mistakes which it makes when talking about statistical modelling…

.

Last Week's Newsletter's 3 Most Clicked Links

• How do you visualize SQL in your head? [Reddit]

• The LLM Critics Are Right. I Use LLMs Anyway.

• If you had to rebuild your entire data platform today from scratch, what stack would you choose? [Reddit]

.
* Based on unique clicks.
** You can find last week's issue #660 here.

Cutting Room Floor

• Size a PostgreSQL server on AWS

• How MVCC and Transactions Work in RocksDB

• How much luck do you need to win the World Cup? - More or Less Investigating the role of random chance in football

• A world still to be mapped: reflections on geocomputation in R: takeaways from the talk and workshop at UseR! 2026

.

Thank you for joining us this week! :)

Stay Data Science-y!

All our best,
Hannah & Sebastian

Issue #660
July 17, 2026

Hello!

Once a week, we write this email to share the links we thought were worth sharing in the Data Science, ML, AI, Data Visualization, and ML/Data Engineering worlds.

And now…let’s dive into some interesting links from this week.

Editor's Picks

• How do you visualize SQL in your head? [Reddit]
  I'm a software engineer that currently helping to build data team, so I work a lot as an "analyst" and build a bunch of data models…During code review, I’m hardly able to tell potential bugs (like joining using wrong key, potential row explosion, etc) at a glance, unlike when reviewing eg: python code. Atm, I’m 50:50 asking claude to generate me a simple viz that could help me trace what happens in each cte transformation. Not always useful, but it slightly helps me. How do you personally tackle this? Do you have an easier mental model that you want to share?…
  

• Football DataPortraits
  It's an impression, but one built entirely from data. Nothing is staged: each match is reconstructed from roughly 1,500 recorded events — every touch, pass, shot and card…

• ARIMA Is Boring, and That Is Why I Still Like It
  ARIMA is old, limited, and a little annoying. I still like it because it makes the assumptions and uncertainty in a forecast harder to ignore…

What’s on your mind

This Week’s Poll:

.

Last Week’s Poll:

.

Data Science Articles & Videos

• The LLM Critics Are Right. I Use LLMs Anyway.
  This week I was at Local-First Conf in Berlin, and the dissonance was everywhere…I spoke at that conference myself, and when I later talked to some of the people, they described the feeling as pretty similar to mine, which is a relief, because I know I am not alone with this…So this article is me trying to describe it. I’ll start by going through all of the fair and valid concerns about using LLMs, the things that would get the big round of applause. Then I will explain what makes me still use LLMs. And I’ll finish up with some of the patterns I found, in the hope that by giving concrete examples, others can step in as well and describe their experiences, so we can all come together and get a better understanding of this dissonance….
  

• If you had to rebuild your entire data platform today from scratch, what stack would you choose? [Reddit]

  Mid-sized company

  Cloud-native

  Batch + streaming

  SQL-heavy analytics

  Some ML workloads…

• Enterprise Haskell at H-E-B
  When I joined H-E-B back in 2018, I was entering a world far removed from any of my previous roles in technology. H-E-B is a retail company first, and the largest privately held company in Texas. Our customers care about full shelves, not the systems behind them, and for decades those systems delivered: rock-solid COBOL mainframes doing their job reliably, day in and day out…
  

• The Little Book of Reinforcement Learning
  This book is a short introduction to Reinforcement Learning, from the basics to applied algorithms…
  

• From Peer Review to Mentorship: My rOpenSci Story

  I first came to rOpenSci in 2022, though at the time I barely knew what it was. I was getting a statistical package of mine ready to submit to the Journal of Statistical Software, and that is how I was pointed toward rOpenSci review: the journal directs authors to rOpenSci’s statistical software standards, so going through the review looked like a convenient step along the way. At the time, my focus was on polishing the software for the journal submission, not on rOpenSci itself…
  

• What is a Smith Chart?
  An interactive introduction to the chart at the heart of RF engineering…


• Latent Thought Flows with Text Compression
  For images, audio, video, and actions, modern generative modeling increasingly shares one recipe: compress the signal into continuous latent tokens, train a generator to map noise to those tokens, and decode them back into the original domain. Language has been the exception…Our core message is simple: text can be compressed into a short sequence of continuous latents that remains useful for generation. This lets language share the same latent-generative recipe as the other modalities, while a text decoder handles surface realization…
  

• A History of Large Language Models
  I trace an academic history of some of the core ideas behind large language models, such as distributed representations, transducers, attention, the transformer, and generative pre-training…
  

• A Bayesian framework for longitudinal EHR and genetic discovery

  Electronic health records (EHRs) provide rich longitudinal disease histories, but existing methods for analyzing these data typically treat diseases in isolation and rarely integrate germline genetics. Here we present ALADYNOULLI, a Bayesian generative framework that jointly models longitudinal EHR diagnoses, age and polygenic risk to recover latent time-varying disease signatures and patient-specific signature loadings…
  

• A ranked-choice election in Maine, USA: Using voting data to understand preferences
  Maine uses ranked choice voting (RCV) in primaries and federal elections, so voters could rank up to six choices for Governor. Using the instant runoff algorithm, candidates were sequentially dropped based on who had the fewest first-choice votes, and ballots were reallocated to each voter’s next-ranked choice…But analyses of the individual ballots cast in the primary reveal a surprising mathematical fact: though she was eliminated before them, Bellows would have defeated either Shah or Jackson in one-on-one elections…This unintuitive fact is a generalization of a well-known feature of ranked choice voting elections: it does not satisfy the Condorcet winner criterion….
  

• Learning from experience instead of curated datasets

  Learning from experience is different from learning from curated datasets. Algorithms that learn from curated datasets can assume that all data is useful for learning. This is usually guaranteed by humans, who collect, clean, and filter raw data so that it is ready to be consumed by a learning algorithm. Experience, on the other hand, does not always contain learnable associations…
  

• All these layoffs have made me question my job search [Reddit]
  Right now I’m at a company that hasn’t done layoffs since maybe the financial crisis. I know how fortunate that is. But if I switch jobs, I could make an extra $50K. So I keep asking myself: is that extra 50K worth the instability that comes with tech jobs right now? What if I join a company and get laid off within a year?…
  

• Users are Humans in Some Context
  If we model a human as a conditional probability distribution: P(action | context),

  then the essence of understanding human behavior—and designing systems for them—lies in unpacking what actually constitutes that context and how the action is selected…

.

Last Week's Newsletter's 3 Most Clicked Links

• A visual introduction to information theory

• Managing/ Dealing with Junior Data Scientists? [Reddit]

• I’m all about ML, but let’s talk about OR

.
* Based on unique clicks.
** You can find last week's issue #659 here.

Cutting Room Floor

• Does playing tennis make you live 9.7 years longer?

• Boolean queries are all you need?

• 100 scientific papers I’ve read in full over the past year

• What Lives? A Meta-Analysis of Diverse Opinions on the Definition of Life

• Domain-Aware Scaling Laws Uncover Data Synergy

.

Thank you for joining us this week! :)

Stay Data Science-y!

All our best,
Hannah & Sebastian

Issue #659
July 09, 2026

Hello!

Once a week, we write this email to share the links we thought were worth sharing in the Data Science, ML, AI, Data Visualization, and ML/Data Engineering worlds.

And now…let’s dive into some interesting links from this week.

Editor's Picks

• A visual introduction to information theory
  We present a visual, intuition-driven guide to key concepts in information theory. We show how entropy, mutual information, and channel capacity follow from basic probability, and how they determine the shortest possible encoding of a data source and the maximum rate of reliable communication through a noisy channel. Our presentation assumes only a familiarity with basic probability theory…
  

• That’s weird! Anomaly detection using R
  This book is about tools and techniques for finding and understanding anomalies. We will begin with some simple data sets containing only one variable, and build up slowly to much more complicated data. We will cover popular but inadvisable methods to identify anomalies (pointing out their shortcomings), as well as more reliable and recommended approaches….

• Rethinking Validation for Spatial Machine Learning: Takeaways from the Talk
  A summary of key points from my keynote and workshop at the Machine Learning for Earth Observation conference in Exeter (2026-06-22 and 2026-06-23)…

What’s on your mind

This Week’s Poll:

.

Last Week’s Poll:

.

Data Science Articles & Videos

• Benchmarking Coding Agents on Databricks’ Multi-Million Line Codebase
  This article shares the results and methodology of the internal coding benchmark we built at Databricks, which evaluates tools on actual coding tasks our engineers performed on the Databricks codebase. Tasks featured edits against a multi-million line codebase covering many popular languages (Python, Go, Typescript, Scala, etc.) and both tasks and solutions were carefully reviewed to ensure accuracy. This isn’t meant to be comprehensive, but the exercise surfaced insights that have already made our engineering team meaningfully more efficient with coding agents. Below, you can see how models and harnesses scored on the overall benchmark…
  

• How much do you actually trust AI output for real reporting? [Reddit]

  Feels like everyone’s using ChatGPT/Copilot for something now, but I can’t tell whether it’s mostly “write me a first draft I’ll review” or something people are comfortable using for production reporting after proper validation. Anyone run into cases where AI-generated code looked completely reasonable but was subtly wrong? How much manual verification do you guys still do vs. just running with it?…

• Matrix Multiplication on Blackwell: Part 1 - Introduction
  In Part 1 (this blog post) we cover what a Matrix Multiplication (matmul) is, its importance for LLMs, and why we need to optimize it. Then we explain what a GPU is, GPU history since Ampere, and finally how to write a simple (not super performant) implementation of matmul on a GPU in 4 lines of Mojo. In part 2, we’ll explain the hardware instructions introduced in Blackwell GPUs, and continue improve on our kernels’ performance to make it leverage the new hardware instructions. As we continue through the blog series, we will incrementally leverage new Blackwell features to improve our matmul implementation until the end of the series where we achieve performance that surpasses that of NVIDIA’s cuBLAS library…
  

• Optimal Transport for Actuarial Science
  These lecture notes introduce optimal transport as a mathematical language for actuarial science. They treat losses, premiums, scores, reserves, capital scenarios, climate losses and lifetime distributions as probability measures that can be compared, transported, averaged, stressed and interpolated. The first part develops the main tools: couplings, push-forwards, discrete and continuous Kantorovich problems, duality, Wasserstein distances, quantile transport, barycenters, entropic regularization and statistical optimal transport. The second part applies these tools to risk measures, Wasserstein robustness, pricing and capital, portfolio drift, reserving cash-flow distributions, climate-prevention diagnostics, reinsurance, dependence uncertainty, capital allocation, distributional fairness diagnostics and longevity risk…
  

• Does additional data always reduce posterior variance?

  Additional data does not always decrease the size of a confidence interval. This post will look at this from a Bayesian perspective. In general, new information reduces your uncertainty regarding whatever you’re estimating. The posterior distribution becomes more concentrated as more data are collected. That’s what happens “in general” but does it necessarily happen every time you get new data? Conceivably if you get surprising data, data that is very unlikely given your current prior, posterior uncertainty might increase…
  

• Markdown styling in R plots and tables
  There is now a well-established – albeit informal – convention in R of using the “label” attribute as a way of storing natural language descriptions of variables, one that is supported by many packages for visualisation such as ggplot2 and some tabulation packages like table…I’m not entirely sure of the history behind this convention, but from what I can tell it seems to have emerged from the haven and labelled packages, which provide valuable tools built using this idea. For the purposes of this post I’ll keep it simple, and define some simple helper functions that make it easy to set, get and modify the labels associated with a data set…


• I’m all about ML, but let’s talk about OR
  There’s a better way! And I feel like nobody talks about it because the Data’s not Big, you’re not Learning Deep things, and there’s nary a chatbot in sight. It’s boring, old operations research, which was something that I guess your university offered, but nobody really knew what it meant. Full disclosure: I still don’t really know what it means. I do know that the job of the data scientist is to bring value to the company, and having some operations and optimization in your toolbelt is quite valuable!…
  

• Samudra 2: A Fast, Cheap AI Ocean Model, Now at the Scale That Matters
  M²LInES’ neural ocean emulator now runs multi-year simulations at eddy-permitting resolution on a single GPU, turning a supercomputer-scale job into one you can run hundreds of times over. That opens the door to faster, cheaper ocean information for shipping, fisheries, energy, insurance, and seasonal forecasting…
  

• Testing of detection tools for AI-generated text

  The study seeks to answer research questions about whether existing detection tools can reliably differentiate between human-written text and ChatGPT-generated text, and whether machine translation and content obfuscation techniques affect the detection of AI-generated text. The research covers 12 publicly available tools and two commercial systems (Turnitin and PlagiarismCheck) that are widely used in the academic setting. The researchers conclude that the available detection tools are neither accurate nor reliable and have a main bias towards classifying the output as human-written rather than detecting AI-generated text. Furthermore, content obfuscation techniques significantly worsen the performance of tools..
  

• patchwork
  The goal of patchwork is to make it ridiculously simple to combine separate ggplots into the same graphic. As such it tries to solve the same problem as gridExtra::grid.arrange() and cowplot::plot_grid but using an API that incites exploration and iteration, and scales to arbitrarily complex layouts…
  

• Knowledge Distillation Explained, Part 1: How Large Models Teach Smaller Ones

  Much of a large model’s capability can be transferred to a significantly smaller model. This is the essence of Knowledge Distillation — a technique where a smaller student model learns from a larger teacher model, aiming to retain most of the teacher’s performance while being faster and cheaper to run…In Part 1 of this series, we trace the evolution of knowledge distillation from Geoffrey Hinton’s foundational 2015 paper through the Transformer era, covering DistilBERT, TinyBERT, and MiniLM…
  

• Managing/ Dealing with Junior Data Scientists? [Reddit]
  I’ve been in the ‘data science’ space for a decade+ or so now. One thing I’ve noticed is that generally - give or take - outside of the elite jobs (<2-3% aka not me and almost certainly not you) the caliber of coworkers has declined drastically. I’m not some fabled data scientist. I wasn’t some GitHub nerd who had everything embroil or terminal wizard nor could I write out the math to a GBM on a blackboard. I’d even forget basic obvious statistics. But I felt like I had common sense. Now I’m a manager/director. I work with data scientists. And I’m just generally freaked out by the absolute lack of basic common sense. This is across the last 7 that I have managed. Examples include…
  

• Distinguishing variables from parameters
  Imagine the following dialog.

  Professor: f is a function of a real variable x that takes a real parameter k.

  Student: What’s a parameter?

  Professor: It’s a constant that can vary.

  Student: Then if it can vary, isn’t it a variable?

  Professor: Sorta, but no not really.

  This conversation plays out over and over, and unfortunately it often ends as it does above, with the student confused. Here’s how I believe the conversation should continue…

.

Last Week's Newsletter's 3 Most Clicked Links

• What is the most underrated skill every data scientist should develop? [Reddit]

• Where to Find the Colors Your Screen Can’t Show You

• HackerRank open sourced its ATS. My resume scored 90/100. Oh wait 74/100. No — 88/100. Actually 83/100.

.
* Based on unique clicks.
** You can find last week's issue #658 here.

Cutting Room Floor

• Learn Stan with brms, Part I

• Building reproducible analytical pipelines with R

• Small Press Distribution Bestseller Lists 2006-2023

.

Thank you for joining us this week! :)

Stay Data Science-y!

All our best,
Hannah & Sebastian

Issue #659
June 26, 2026

Hello!

Once a week, we write this email to share the links we thought were worth sharing in the Data Science, ML, AI, Data Visualization, and ML/Data Engineering worlds.

And now…let’s dive into some interesting links from this week.

Editor's Picks

• Where to Find the Colors Your Screen Can’t Show You
  There are colors that I want to show you, but I can’t. They exist in the real world. You probably saw some of them today, but I can’t show them to you on a screen. A digital photograph can’t capture them, and your screen can’t display them. No game you’ve ever played has contained them. Unless you have specialized equipment, they are entirely absent from the digital world…
  

• Finding the Best Dog Treat with Statistics: A Greyhound, five treats, and a Bradley-Terry model
  Bebop, my 83lb, 33 inch tall, Greyhound, loves three things: running fast, following me around the house, and treats. Whether it’s a chew treat, pizza out of a child’s hand who strayed too far from a party, or a small tray of cat food, he has a nose for what he likes and the athleticism to give him a fair shot at getting it. I’ve watched him eat for years, so it was upsetting to realize I don’t know what his favorite snack is, and can’t easily ask him. Fortunately for Bebop’s palate, the Bradley-Terry model gives us a way to figure out a “strength” of treat from pairwise comparisons…

• Generative Artificial Intelligence creates delicious, sustainable, and nutritious burgers
  Using burgers as a model system, the generative AI rediscovers the classic Big Mac without explicit supervision and generates novel burgers optimized for deliciousness, sustainability, or nutrition. Compared to the Big Mac, its delicious burgers score the same or better in overall liking, flavor, and texture in a blinded sensory evaluation conducted in a restaurant setting with 101 participants; its mushroom burger achieves an environmental impact score more than an order of magnitude lower; and its bean burger attains nearly twice the nutritional score. Together, these results establish generative AI as a quantitative framework for learning human taste and navigating complex trade-offs in principled food design…

What’s on your mind

This Week’s Poll:

.

Last Week’s Poll:

.

Data Science Articles & Videos

• HackerRank open sourced its ATS. My resume scored 90/100. Oh wait 74/100. No — 88/100. Actually 83/100
  How hiring is becoming a luck filter…Here a quick rundown on how the tool works: Your PDF gets parsed into text. An LLM is called six times to extract structured information — your basics, work history, education, skills, projects, awards. It pulls your GitHub profile, scans your top repos, appends them as extra context. Then everything gets fed into the LLM at once to be graded…
  

• What is the most underrated skill every data scientist should develop? [Reddit]

  Beyond Python, machine learning, and statistics, which skill has made the biggest difference in solving real-world data science problems and delivering business value?…

• Can AI Become a Real Data Scientist? | Gaël Varoquaux on scikit-learn, Probabl & Scientific Judgment
  In this episode of Creative Difference, Maxime Gabella speaks with Gaël Varoquaux, co-founder of scikit-learn and Probabl, about the future of AI, statistical learning, and scientific judgment. They discuss the difference between generative AI and statistical machine learning, why current AI agents can write code but still struggle with rigorous data analysis, and why tools like scikit-learn and Skore may become essential for trustworthy AI systems. The conversation explores memorization, generalization, uncertainty, data leakage, evaluation, human-in-the-loop science, and the role of a “statistical harness” for AI agents. It also moves into deeper questions: why real-world science is harder than mathematics, why data collection and intervention still matter, how categories shape reality, and what creativity means for humans and machines…
  

• Vibe Coding vs. Vibe Engineering: How Systems Scale Without Collapsing
  What is the difference between vibe coding and vibe engineering? Learn how modern software teams evolve from rapid experimentation to enterprise-grade systems without drowning in technical debt or sliding into technical bankruptcy…
  

• Why Adjusted Regression Coefficients Are Less Descriptive Than They Look

  It is common for researchers to investigate “factors associated with” an outcome by collecting many candidate variables, entering them together into one multivariable regression, and reporting the ones whose mutually-adjusted coefficients reach significance (Lewer et al., 2025). In this interactive article I explain why that practice misleads — not because of the usual problems of causal inference or multiple comparison, but even in descriptive studies, where researchers are genuinely interested in associations…
  

• From Julia to Rust: a differentiable tensor stack for scientific computing in the agentic AI era
  The Rust ecosystem has changed a lot in the last few years. crates.io went from 602 crates in 2015 to roughly 210,000 in 2026 (data). For dense linear algebra there is faer; for GPU kernels, CubeCL; for generic numerics, num-traits and num-complex. There are also libraries at nearby layers: ndarray for arrays, nalgebra and faer for linear algebra, Burn and candle for deep learning, and numr for a NumPy-style array API. What we needed was the layer between them: a scientific-computing tensor stack with column-major storage, dynamic shapes, eager and traced autodiff, einsum, FFT, CPU/CUDA backends, and extensible operations. That is what tenferro-rs is for…This post explains why we are building it, and why we chose Rust now that code is no longer written only by humans…


• Gribouille - Create elegant graphics with the Grammar of Graphics for Typst.
  Compose charts by layering data, aesthetic mappings, geoms, scales, and themes. The same grammar as ggplot2 and plotnine, drawn natively in Typst…
  

• Open-source geo-experiment tools are not interchangeable
  We ran 32,000 simulated experiments across four common marketing scenarios to benchmark four leading open-source geo-experiment tools. Because we use synthetic data where the true campaign effect is known in advance, we can measure how well each tool recovers it. These four tools are often treated as interchangeable, and they are not. Where they diverge — sharply, at times — is in how they handle uncertainty: how often their confidence intervals contain the true incremental effect, how often they declare winning results that aren’t real (false positives), and how often they come back inconclusive when a real incremental effect exists (false negatives)…
  

• Ten years of ClickHouse in open source

  ClickHouse was released in open source on Jun 15 2016, ten years ago. Since then, it became the most popular open source analytical database with more than 2000 contributors…
  

• What happens when you run a CUDA kernel
  Here’s a simple CUDA program. It adds two vectors…Compiled for an RTX 4090, and launched, it does correctly work out that 1 + 1 = 2, a million times…Telling you that involved tens of millions of CPU instructions, a couple of device files, nine hundred ioctls, and one memory-mapped doorbell register. In this post, we’ll follow this one kernel from the code down to the warps, and back up to the answer…
  

• Map Clustering is Not My Favorite

  Friends, this post has been in the making for maybe, 20 years. Google Maps (and me) are (at least) that old. And mapping, of course, existed before that; those were the days. But ~~~20 years ago, or like a year after that, something appeared on our collective mapping radars that I have not been able to shake off since - and which I haven’t kvetched about in writing, ever, for everyone to “enjoy”. Enjoy! It’s a Greg Rant! Amazing… What happened ~~~~20 years ago, after Google Maps was hoisted on us all, was that “people wanted to see more than 1 point on a map” and sometimes that meant 100 points, or a thousand! And things were not easy or good then. I mean, 1000 points, that’s a LOT. But then, who said 1000 points (was it you?) was too much to show? Well… browsers…
  

• Data Directors - what’s your next step? [Reddit]
  For anyone who has had a director of data or data director title in the past - where are you now? Similar role at a different company? Same role? Eventually C-suite? What’s the plan?…
  

• Zen and the Art of AI Research (temperament >> talent)
  So you want to do AI research? It’s true that no one really teaches you how. Not directly, anyway. The way to get started is pretty simple: some combination of (i) reading and (ii) building stuff. You can’t do one without the other. You become a researcher through the combination. It turns out the process of becoming a great researcher is not unlike learning to meditate…

.

Last Week's Newsletter's 3 Most Clicked Links

• Surprising lessons from my research scientist job search

• The PCA Mistake I Made During My PhD (and How to Avoid It in R)

• United Kingdom prime ministers

.
* Based on unique clicks.
** You can find last week's issue #657 here.

Cutting Room Floor

• Generative AI for material design: A mechanics perspective from burgers to matter

• Faster KNN search in Manticore: 2-pass HNSW, batched distances, and AVX-512

• Hnswlib - Header-only C++/python library for fast approximate nearest neighbors

• Working With AI: A Concrete Example

• Matrix Orthogonalization Improves Memory in Recurrent Models

• The Doorman Fallacy In Action

• DuckDB Internals: Why is DuckDB Fast? (Part 1)

• the-stats-duck v0.6.0 — statistics that live in your SQL

• Small Molecules Have More Information Per Atom Than Biologics

.

Thank you for joining us this week! :)

Stay Data Science-y!

All our best,
Hannah & Sebastian

Issue #657
June 25, 2026

Hello!

Once a week, we write this email to share the links we thought were worth sharing in the Data Science, ML, AI, Data Visualization, and ML/Data Engineering worlds.

And now…let’s dive into some interesting links from this week.

Editor's Picks

• United Kingdom prime ministers
  UK has had a spurt of prime ministerial turnover in the past decade or so, but it’s by no means unprecedented. I download data from Wikipedia and try several ways to visualise that turnover…

• How The Heck Do Synthesizers Work? (An Interactive Exploration)
  Synthesizers have remained a staple of modern music. From iconic video game soundtracks to Hans Zimmer’s scores, from Radiohead to the Stranger Things theme. We hear more synthesized music than we even realize. So how do they actually work?…

• Surprising lessons from my research scientist job search
  There are two recent blog posts from Alisa and Silvia, both CS PhD students, on how they prepared and got into frontier labs such as OpenAI and Google Deepmind. I highly recommend them, and after seeing the reactions on Twitter, I want to share a different angle: what surprised me during my own research scientist job search…

What’s on your mind

This Week’s Poll:

.

Last Week’s Poll:

.

Data Science Articles & Videos

• The PCA Mistake I Made During My PhD (and How to Avoid It in R)
  A few years into my PhD, I ran a principal component analysis on a dataset I’d spent six months collecting, looked at the scree plot, picked “the number of components that looked right,” and moved on. A reviewer later asked me to justify that number. I couldn’t. Not with anything more rigorous than “the elbow looked like it was there.”…
  

• It turns out Analytics was a great career to go into even in a world with AI [Reddit]

  Maybe two or three years ago I lamented the fact I had never gone into software development in spite of the fact I probably had the coding mindset for it, regretting the tedious and stressful aspects of Analytics as well as lower overall pay. Now with AI leading to massive layoffs and / or reduce hiring in software development and other Engineering fields, I’m thinking Analytics was a good field to specialize in since it has that sweet spot of being just close enough to the business and just close enough to the tech side that it is hard to automate away via AI. Furthermore, I think demand for analysts in general to understand data and accommodate reporting changes will also increase if AI is accelerating software changes and changes to data models and systems…

• minFLUX - A hackable implementation of FLUX diffusion models
  A simplified educational PyTorch implementation of FLUX.1 and FLUX.2 diffusion transformers (DiT) by Black Forest Labs. Built for understanding rectified flow matching, joint attention, and the key design choices behind FLUX with verifiable line-by-line source mappings to the official codebases…
  

• Data-driven discovery of dynamical models in biology
  In this review, we survey approaches for model discovery in biological dynamical systems, focusing on three methodological families: regression-based methods, network-based architectures, and decomposition techniques. We compare their ability to address three core goals: forecasting future states, identifying interactions, and characterizing system states. Representative methods are applied to a common benchmark, the Oregonator model, a minimal nonlinear oscillator that captures shared design principles of chemical and biological systems. By highlighting strengths, limitations, and interpretability, we aim to guide researchers in selecting tools for analyzing complex, nonlinear, and high-dimensional dynamics in the life sciences…
  

• Scaling Laws, Carefully

  Scaling laws are one of the most critical empirical findings in deep learning. The observation is simple in form: the training loss L decreases predictably as we scale up model size N, dataset size D, and compute C, following a power-law curve, which appears as a straight line on a log-log plot. We can view scaling laws as a framework for describing the relationship between compute, loss, model size and data; at its core, it is about how to allocate precious compute optimally between N and D. This predictability makes scaling laws highly valuable in practice. A common workflow is to fit scaling laws on a handful of small runs and then extrapolate to estimate the token and compute requirements for larger models….
  

• ML Foundations (prerequisites) for Post-Training | RLHF Book Course, Lecture 0
  In this video I try to cover a bunch of math, LLM training fundamentals, and probability concepts that come up again and again in post-training content (and this book). We cover things like the role of mid-training, definitions of KL, entropy & cross-entropy, getting LM probabilities from a sequence, etc…


• tsauditor - data quality auditing library for time-series tabular data in financial and sensor domains
  A data-quality auditing library for time-series tabular data, with a focus on financial and sensor domains. tsauditor scans a DataFrame and returns a structured report of structural problems, anomalies, and — its core contribution — data-leakage between features and the prediction target…
  

• General-purpose large language models outperform specialized clinical AI tools on medical benchmarks
  Specialized clinical artificial intelligence (AI) tools are entering medical practice at scale1,2. These proprietary large language model (LLM)-based tools promise superior clinical performance to general-purpose frontier LLMs as a result of domain-specific training or retrieval-augmented generation (RAG)3. Yet, their architectures, base models and training pipelines are not public…This study is an independent, quantitative comparison of clinical AI tools against frontier LLMs using real-world physician queries from the course of care. Clinical AI tools lagged behind frontier models on every evaluation: knowledge, expert alignment and real-world clinical use across multiple dimensions…
  

• Moneyball for Physical AI - A Scaling Law Perspective for Marginal Utility per Dollar

  This essay builds a framework for the marginal utility of data, and uses it to discuss value accrual in Physical AI. We take the perspective of the scaling laws that guide how loss behaves with data, and the unit economics that govern what a dollar of data is worth. Together they give an approximate marginal utility per dollar, the on-base percentage of physical AI…
  

• Five Eras in the Evolution of Probability and Statistics
  Statistics isn’t a new thing. It dates back at least fifty centuries beginning as counts in the form of tally marks for keeping track of crops, animals, people, and time. From there, it evolved with the demands of government and business, supported by academic inquiry and the growth of technology…
  

• Interested in learning more about digital archives?

  Interested in learning more about digital archives? Here are some resources to get you started! Most are targeted at a general audience, but some of them are more technical. I’ve included descriptions to help you navigate. Let me know if you have any topics you’d like me to cover here!..
  

• Anyone actually believe dashboards are going away? [Reddit]
  keep seeing this take that ai agents are going to replace dashboards entirely. like why even look at a chart when the ai can just tell you "stop spending on this channel" or whatever. and i get the appeal of that but i think it misses something pretty fundamental about how people actually make decisions…
  

• Data for fitting models versus data for predicting from models
  Answering a question that came up from a student recently. Say you have 20 surveys of reef fish biomass at different locations. Then you also have gridded data with environmental covariates. The gridded data is for all reefs everywhere. The goal is to predict fish biomass at all reefs everywhere. Here’s an older post that walks through the steps in R with older packages (you will want to update raster to terra, everything else should work). The statistically correct workflow would look like this...

.

Last Week's Newsletter's 3 Most Clicked Links

• 2026 Tech Stack at your Job [Reddit]

• Free SQL→ER diagram tool, runs in the browser, nothing uploaded

• Running local models is good now

.
* Based on unique clicks.
** Please take a look at last week's issue #656 here.

Cutting Room Floor

• Histories of Digital Culture

• Introducing Un-0: Generating Images with Coupled Oscillators

.

Thank you for joining us this week! :)

Stay Data Science-y!

All our best,
Hannah & Sebastian

Issue #656
June 18, 2026

Hello!

Once a week, we write this email to share the links we thought were worth sharing in the Data Science, ML, AI, Data Visualization, and ML/Data Engineering worlds.

And now…let’s dive into some interesting links from this week.

Editor's Picks

• Free SQL→ER diagram tool, runs in the browser, nothing uploaded
  Paste a SQL schema (CREATE TABLE statements) → get a clean, interactive ER diagram. Open source and 100% local — it runs entirely in your browser, so your schema never leaves your machine. No server, no signup, no upload…

• Preschoolers search semantic networks in a broader and more variable way than adults: Implications for hypothesis generation
  We find that adults show greater dependencies between sequential guesses than preschoolers, and generate a less diverse set of options. These findings may support the idea that development can be viewed as analogous to simulated annealing strategies in machine learning that start “hot” (in early childhood), generating wider and more variable searches, and eventually cool (in adulthood) to generate narrower searches…

• New CRAN Packages: signal or noise?
  CRAN continues to be the most accessible repository for statistical knowledge on the planet, and the number of new packages being accepted by CRAN is growing faster than ever. But, is the R community really benefiting from this new growth?…

What’s on your mind

This Week’s Poll:

.

Last Week’s Poll:

.

Data Science Articles & Videos

• The 90-year-old idea behind JEPA models: Canonical Correlation Analysis (CCA)
  Harold Hotelling’s 1936 Canonical Correlation Analysis (CCA) [modern terminology, “CCA is used to find a common signal among two large matrices”] forms the theoretical and intuitive foundation for modern embedding prediction techniques, including JEPA models…
  

• 2026 Data Science Tech Stack at your Job [Reddit]

  What is your current tech stack at your job?

• A Beginner’s Guide to Robotics Hardware
  Building an open-source robot begins in a way that is familiar to assembling IKEA furniture…The similarity ends once the robot is powered on. A bookshelf is designed to stay exactly as assembled, whereas a robot has to move while remaining correct about its own position and the state of its surroundings…When designing and building robots, this need for correctness both numerically and temporally is a key consideration…The rest of this post looks at how that difference manifests, using a common framing in robotics that divides the hardware into three parts: the movement, the body, and the sensor…
  

• PyData London 2026 Talks
  All the talks from the PyData London 2026 are now available…
  

• Data Engineering Acquisitions (2022-2026)

  Consolidation in the Data Engineering market is happening quickly. Tools from the Modern Data Stack get unified into bigger Data Platforms. This note highlights the latest acquisitions across data engineering. It serves as an overview of the latest consolidations. Find attached the acquisition overview from 2022 to today…
  

• The software industry: annealing, but wrong
  In recent months I’ve heard of several teams with an interesting policy: each pull request should be no more than a few files, and no more than a certain number of lines (say 500). And do just one thing and do it well. And be easy for a human to review. And be fully tested by the test suite…And often, the results are good. Sure, splitting a single 6000-line feature or fix into twelve 500-line PRs is more work, but each of those PRs is surely easier to review. And you can git bisect them when there’s a bug! And maybe revert the individual change that broke something. ...and also cause 12x as many context switches for your reviewers as they review each one sequentially. But that’s just the cost of software quality! Right?…


• AI and Survey Sampling Problems
  My previous post discussed the performance of the artificial intelligence (AI) interface Gemini on undergraduate statistics problems. Now let’s look at how Gemini answers some of the problems in my sampling textbook (Lohr, 2022), and talk about how Gemini could help students learn sampling…
  

• Test Doubles Taxonomy for R: Dummy, Stub, Spy, Mock, Fake
  You might call them all “mock”…Mock the database. Mock the API. Mock the function. The word becomes a catch-all for any test double, any object you substitute for a real dependency in a test. Lumping them together makes it harder to choose the right tool, and the wrong choice leads to brittle, misleading tests. There are five distinct types, each with a specific job. Knowing which is which is how you stop writing tests that do the wrong thing…
  

• Fractional Brownian Motion

  A Brownian Motion (BM), without the “fractional” part, is a motion where the position of a given object over time changes in random increments…A Fractional Brownian Motion is a similar process in which the increments are not completely independent from each other, but there’s some sort of memory to the process…I believe fBM is not necessarily a well understood mechanism. So this article describes how it functions and their different spectral and visual characteristics for various values of their main parameter H, backed with some experiments and measurements…
  

• Running local models is good now
  I’ve been working with local models since they came out, and finally, they’re surprisingly good now…
  

• Snapshot Testing in R: Beyond Screenshots

  In this post I want to walk through using snapshot testing for what it is good for, and the practices that make it efficient…
  

• What algorithm is surprisingly new? [Reddit]
  Other than any AI stuff, I’m talking about the types of algorithms you learn about in any standard Data Structures and Algorithms University course…I’m surprised that alot of these algorithms were actually invented HUNDREDS of years ago…
  

• The Ghost Couple: Correlated LLM Name Priors and Their Haunting of the Web and Academic Publishing
  These names do not exist. Elena Vasquez and Marcus Chen have appeared as volcano experts, astronauts, thriller protagonists, podcast hosts, and academic co-authors across hundreds of independently produced AI-generated documents, never having lived. We show that llms do not merely default to high-probability individual names when generating fictional experts: they produce correlated character ensembles, pairs and trios whose co-occurrence rates far exceed chance and are consistent across independent generations. These priors are model-family-specific (Claude: Elena Vasquez + Marcus Chen + Amara Okafor; Gemini: Aris Thorne + Lena Petrova; GPT: Elara Voss with no fixed partner)…

.

Last Week's Newsletter's 3 Most Clicked Links

• Forecasting: Principles and Practice, the Pythonic Way

• My unvarnished guide to solution engineering

• An Overview of Modern AI Robotics from First Principles

.
* Based on unique clicks.
** Please take a look at last week's issue #655 here.

Cutting Room Floor

• 2026 Rousseeuw Prize for Statistics Awarded to R Core Team for Transforming Statistics Computing Worldwide

• Learning Amino Acids Part 1: Non-Polar Amino Acids, Rodrigues Rotation, and Lennard-Jones Potential

• Gambling provides a gentle rocking of the emotions to put you in a pleasant baby-like state

• The Caitlin Clark Effect

• Formal methods and the future of programming

• Crunch Big Data on Your Laptop With Polars Streaming

.

Thank you for joining us this week! :)

Stay Data Science-y!

All our best,
Hannah & Sebastian

Issue #655
June 11, 2026

Hello!

Once a week, we write this email to share the links we thought were worth sharing in the Data Science, ML, AI, Data Visualization, and ML/Data Engineering worlds.

And now…let’s dive into some interesting links from this week.

Editor's Picks

• An Overview of Modern AI Robotics from First Principles
  There is a deceptively simple way to describe what physical AI is all about, a way in which anyone with a STEM background will intuitively understand. Like all other AI models, a model which controls a robot is also a function. It takes in observations (camera pixels, joint angles, the felt resistance of a gripper, etc) and it outputs actions, the next set of positions and torques for its motors…If you’ve ever trained a model that maps inputs to outputs, you can already grasp the shape of the problem. The interesting part is what happens when you take this familiar shape and drop it into a moving, active world…This sounds like ordinary machine learning, and for a while you can pretend it is. But robotics introduces a third axis that classic ML never had to respect: inference time…

• My unvarnished guide to solution engineering
  Nowadays I feel more or less comfortable interacting with customers. But I was awful at first. I know because one of the cofounders gave me harsh feedback after a call with our first serious customer. I still remember slamming the lid of my computer when we debriefed. What I perceived as harsh feedback at the time turned out to help me grow quickly…I used to be a regular data scientist assigned to internal projects. Talking to prospects and customers got me out of my comfort zone. You owe them a service, and they expect you to deliver something. If something goes wrong they’ll go above your head to your founders, at which point you start feeling the heat. It can be quite harsh. But it can also be rewarding when things go well…

• Navier-Stokes fluid simulation explained with Godot game engine
  Let me start with the mathematical description of what we will do in this blog post. This description might sound daunting, but don’t worry - we’ll explain everything as we go. Here goes: we will simulate fluid flow by moving a scalar density field through a vector velocity field. We’ll simulate velocity diffusion and advection as well as density diffusion and advection. Then we will add velocity projection with the goal of making the fluid obey the law of mass conservation - which will happen by balancing divergence with a pressure field. We will use bilinear interpolation and Gauss-Seidel relaxation for approximating values where needed…

What’s on your mind

This Week’s Poll:

.

Last Week’s Poll:

.

Data Science Articles & Videos

• The Anti-Scaling Law in Biology, and Why AI Could Make Crowding Worse Before Making Drug Development Better
  One of the main reasons for the tech community’s optimism is the scaling-law. Once you demonstrated 0-1, you can do 1-100 much quicker. The internet, social media, and so on…In biology and drug development, I think there is a mirror image, the anti-scaling law. Because of that, here’s my contrarian view: AI could make crowding in drug development worse, before making it better. And that’s my perspective as a genuine believer in the transformative power of AI, and an AI practitioner who used $14,000 worths of AI tokens in the past 2 months…
  

• What is there besides Frequentist and Bayesian stats? [Reddit]

  I am wondering whether there are lesser known statistical paradigms. like most people, I was first acquainted with the Frequentist framework, and later got introduced to Bayesian stats. I really like the way this made me reconsider some of what I thought were basic assumptions, so now I’m wondering what the next thing could be? Are there any other branches/frameworks which are not as well known?…

• Forecasting: Principles and Practice, the Pythonic Way
  This textbook is based on Forecasting: Principles and Practice (3rd ed) and is intended to provide a comprehensive introduction to forecasting methods and to present just enough information about each method for readers to be able to use them sensibly. We don’t attempt to give a thorough discussion of the theoretical details behind each method, although the references at the end of each chapter will hopefully fill in many of those details…
  

• The Simplest Learning Machine, Pt.2
  In the previous article I outlined the concept of the Simplest Learning Machine. It’s an imaginary algorithm that uses one byte of persistent memory and learns to predict something about a stream of binary events…Can we actually write something like that? How would it work?…One semi-obvious thing we can learn is the rate of positive events in the stream. This would give us some predictive power, as long as that rate is different from 50%. A bit of a stretch to call this “machine learning”, sure, but I’ll get to the questions of usefulness later…
  

• On Training Data for Bio AI Models

  As we advance biological foundation models, which lessons from LLM data curation transfer, and which need rethinking?…
  

• Our goodpractice Package Has New Superpowers
  The goodpractice package has been recommended by rOpenSci since it was first started just over 10 years ago by Gábor Csárdi. We used to ask our editors to manually run goodpractice on all packages submitted to software peer-review, and then to ask authors to fix any notable issues flagged by the package…We’re really pleased to share that we’ve recently rolled out a host of updates and extensions to the package. These make it both easier to use, and more powerful…I think that is the aspect that I found most surprising: That the use of Claude made our collaboration feel less technical, and therefore somehow even more human. And that gave us the ability to work though 70 pull requests representing over 100 new checks, all ready for everybody to use…


• The Warehouse
  The Problem

  With over 23,000 packages on CRAN alone, finding the right package for your task is overwhelming:

  • Searching by keywords often misses relevant packages

  • No easy way to compare similar packages

  • Quality indicators are scattered or missing

  • GitHub-only packages are hard to discover

  The Warehouse Solution provides:

  • Function-first search: “estimate serial interval” → find all relevant packages

  • Quality scores: Automated assessment of tests, documentation, and maintenance

  • All sources: CRAN, GitHub, Bioconductor in one place

  • Community reviews: Real user experiences and recommendations

  • Smart categorization: Browse by what packages actually do…
    

• Robot Learning: From Fundamentals to Foundation Models
  This course provides a comprehensive introduction to modern robot learning, combining classical techniques with the latest advances in large-scale models: Students will start by learning the fundamentals of imitation learning, reinforcement learning, and policy optimization, and gradually progress to advanced topics including Vision-Language-Action (VLA) models and foundation models for robotics The objectives of this course are:

  • Understand the core principles of imitation learning, reinforcement learning, and policy learning.

  • Implement basic robot learning systems in simulation and on real robots.

  • Explore state-of-the-art Vision-Language Action and foundation models for robotics.

  • Design and evaluate scalable robot learning pipelines integrating perception, control, and multi-modal reasoning…
    

• ML Job Interviews: The Ultimate Guide

  How I found a Research Scientist role after a PhD in Machine Learning…My process was, overall, successful: I received offers from every company I completed interviews with including: DeepMind (which I accepted), Isomorphic Labs, Cohere, Meta, and a startup in stealth. A few caveats to the first claim: Anthropic, Mistral, and TeslaAI got back to me too late and I didn’t complete those processes. ReflectionAI, the one genuine rejection: they didn’t like me for the RS role but switched me to their Engineering track instead…
  

• stata-mpl - Give your matplotlib and seaborn charts the Stata 19 look
  Give your matplotlib and seaborn charts the look of Stata 19 (the stcolor scheme, Stata’s colorblind-friendly default). Calibrated against the official SVG files exported by Stata 18/19…
  

• Avian Visitors

  I mounted a tiny microphone on my apartment balcony to listen for any birds passing by and built a site to collage them as they’re heard…so I’ve thrown together this short writeup for any of you who want to monitor any avian visitors that may be passing by your own place. It’s short and sweet for now in an attempt to get something out quickly, but this work is part of a longer chain of bird-tangent projects i’ll write something up about soon!…
  

• Ask HN: What are tools you have made for yourself since the advent of AI?
  Ask HN: What are tools you have made for yourself since the advent of AI?…
  

• Why Academics Should Use AI for Writing: A Case Study
  I violently dislike the idea of AI taking over my writing. My writing is my own, and having it done by AI makes the final product lose its soul. Also, whenever I have used AI to write several paragraphs independently (which I admit to doing for bureaucratic tasks) I ended up rewriting most of it anyway. However, over the past year or so I have become increasingly impressed with what AI can do, and rather than talk about this in abstract terms I would like to present you with a concrete demonstration that shifted my opinion a great deal…

.

Last Week's Newsletter's 3 Most Clicked Links

• How to Build a Simple, Bulletproof Data Pipeline

• supertree - Interactive Decision Tree Visualization

• Beyond the Semantic Layer: Building a Context Layer for the Agentic Era

.
* Based on unique clicks.
** Please take a look at last week's issue #654 here.

Cutting Room Floor

• When is detecting AI-generated text worthwhile?

• Lines of Code Got a Better Publicist

• BayesMultiMode: Bayesian Mode Inference in R

• Cognitive Endurance as Human Capital

• DiffusionBlocks: Training Neural Networks One Block at a Time

• Data from 66,000+ practice sessions. How much does the typical musician actually practice? [Reddit]

• What is the most common reason data science projects fail to deliver business value? [Reddit]

.

Whenever you're ready, 3 ways we can help:

1. Go deeper each week (paid subscription)
   Get 3 additional posts per week designed to help you:

   • Statistics → understand the math behind ML

   • AI Agents → build with modern AI tools

   • Career → become more valuable at your job

   👉 Upgrade for $10/month — cancel anytime
   

2. Looking to get a job?
   A practical guide to landing your first (or next) data science role, based on thousands of reader questions.
   👉 Check out our “Get A Data Science Job” Course
   

3. Promote your organization/project/event to ~68,500 subscribers
   Sponsor this newsletter and reach a highly engaged data science audience (30–35% open rate).
   👉 Reply to this email to learn more

Thank you for joining us this week! :)

Stay Data Science-y!

All our best,
Hannah & Sebastian

Issue #654
June 04, 2026

Hello!

Once a week, we write this email to share the links we thought were worth sharing in the Data Science, ML, AI, Data Visualization, and ML/Data Engineering worlds.

And now…let’s dive into some interesting links from this week.

Editor's Picks

• How to Build a Simple, Bulletproof Data Pipeline
  In most organizations, a lot of value can be delivered with a setup that avoids high costs and unnecessary complexity. The most common scenario is not real-time streaming or exotic architectures. It is daily extraction from one or more transactional systems backed by a relational database…In this article, I want to walk through a simple but realistic example and show how a few design decisions, even when they look basic, can make a meaningful difference in robustness, operability, and long-term maintainability…

• AI for Bio has a Fuzzy API problem
  “AI for bio” is getting hot again. Given the excitement in the current moment, I thought I’d share a bit about what actually makes biology uniquely hard as an application domain for machine learning. The reason is not simply that biology is complicated, though it obviously is. ML is good at many things that are complicated. The deeper reason is that drug discovery does not have the kind of clean feedback loops and clean interfaces that made modern ML so powerful elsewhere…

• Gaussian Point Splatting
  We propose Gaussian point splatting, a stochastic method to render Gaussian splats that scales extremely well to scenes with many Gaussians. Our core idea is to sample pixel-sized, opaque points from the Gaussians and to splat them to a framebuffer using 64-bit atomics. Through parallel programming primitives, we achieve an even distribution of the workload across millions of threads…

What’s on your mind

This Week’s Poll:

.

Last Week’s Poll:

.

Data Science Articles & Videos

• State of London Report, 2026
  The State of London report brings together an array of data about how London is performing across its economy, society and environment. Published annually by the GLA’s City Intelligence unit, it provides an up‑to‑date, data‑led picture of life in the capital, drawing on a wide range of official and administrative sources. The report highlights trends and patterns and offers a shared evidence base to support debate and decision making…
  

• What is considered basic R? [Reddit]

  I have a job interview coming up and they want someone who knows basic R, I think I do have it, but what is your opinion on what it entails?…

• BRIDGE (Benchmark of Regional & International Data for Global Evaluation) is the first independent Global South ASR benchmark
  BRIDGE (Benchmark of Regional & International Data for Global Evaluation) is the first independent Global South ASR benchmark evaluating 15 global models across 22 languages on a first-of-its-kind 7 metric stack…
  

• supertree - Interactive Decision Tree Visualization
  supertree is a Python package designed to visualize decision trees in an interactive and user-friendly way within Jupyter Notebooks, Jupyter Lab, Google Colab, and any other notebooks that support HTML rendering. With this tool, you can not only display decision trees, but also interact with them directly within your notebook environment. Key features include:

  • ability to zoom and pan through large trees,

  • collapse and expand selected nodes,

  • explore the structure of the tree in an intuitive and visually appealing manner…
    

• A Primer in Post-Training Reasoning Data: What We Know About How It Works

  Work on post-training reasoning data has grown rapidly, yet this literature remains scattered across dataset papers, reinforcement-learning recipes, reward-model studies, benchmarks, and frontier system reports. This paper is the first primer to synthesize over 150 key public studies and system reports on post-training reasoning data. We organize the field around four questions: what data objects exist, what makes them useful, how they are constructed, and how they scale. Together, this organization provides an attribution framework for future reasoning-data releases and post-training recipes…
  

• Don’t know where your data is from? Bayesian modeling for unknown coordinates
  An especially strong motivating case for the usage of spatial probability models comes from the mining industry. During exploration for mineral resources, prospectors will take geologic samples by drilling holes and examining the resulting material for presence or concentration of valuable ores. These data typically show strong spatial correlation, but constructing a fully-detailed geophysical model is at times infeasible as we are able to observe very little of the underground conditions, though the advent of remote sensing techniques like ground-penetrating radar and gravimetry has dramatically improved our ability to characterize Earth’s subsurface. To address this challenge, we would like to construct a probability model which uses nearby data to predict a variable of interest at a new location…


• Why I think panic about local impacts of data centers is just a panic
  In the last year of following increased local resistance to data centers being built, I’ve listened to lots of recorded testimony at town halls, read through countless comments and articles where people argue for why data centers are so uniquely evil and shut down anyone defending them as shills for AI companies…whenever I look into where people are actually getting their ideas about the hundreds of other data centers being built, the source always leads back to some confused misreading of local reporting, a wild calculation error, a bad game of telephone, or a wildly misleading article…
  

• Beyond the Semantic Layer: Building a Context Layer for the Agentic Era
  A context layer puts your warehouse schema, joins, metric definitions, and business knowledge in one reviewable place so data agents query governed context instead of guessing field names. A look at how it works, and at ktx, the open-source context layer…
  

• Jensen-Shannon Divergence

  How different are two discrete or binned probability distributions on the same support?…
  

• Fluid Simulation for Dummies
  I wrote my Master’s thesis on high-performance real-time 3D fluid simulation and volumetric rendering. The basics of the fluid simulation that I used are straightforward, but I had a very difficult time understanding it. The available reference materials were all very good, but they were a bit too physics-y and math-y for me. Unable to find something geared towards somebody of my mindset, I’d like to write the page I wish I’d had a year ago. With that goal in mind, I’m going to show you how to do simple 3D fluid simulation, step-by-step, with as much emphasis on the actual programming as possible….
  

• A Hitchhiker’s Guide to ML Training Infrastructure

  Hardware has made a huge impact on the field of machine learning (ML). Many of the ideas we use today were published decades ago, but the cost to run them and the data necessary were too expensive, making them impractical. Recent advances, including the introduction of graphics processing units (GPUs), are making some of those ideas a reality. In this post we’ll look at some of the hardware factors that impact training artificial intelligence (AI) systems, and we’ll walk through an example ML workflow….
  

• Weaponized phrases in Data science Teams [Reddit]
  “No free cycles” / “Empty plates”…“We need to focus on the low-hanging fruit”…"Be a go-getter, don't get stuck"…"Let's optimize our sprint velocity"…“You’re making this more complicated than it is”…"We need to relentlessly prioritize"…"I need you to own this initiative"…"Let's take this offline" / "Parking lot this"…"We need to leverage AI to unlock enterprise value"…"We're like a family here"…
  

• Agricultural Field Boundaries, Mapped Globally for the First Time
  For the first time, every agricultural field on Earth has a boundary on the map. Taylor Geospatial funded and co-developed this work with Microsoft AI for Good Lab because we believe GeoAI should work everywhere, not just in the data-rich regions where labeled training data is abundant. Today, it’s publicly available for everyone to benefit from…

.

Last Week's Newsletter's 3 Most Clicked Links

• What DS job market trends are you seeing? [Reddit]

• 6 LLM Prompting Techniques for Data Scientists and Engineers in 2026

• I am faking my way through a Data Analyst role with AI, how do I actually learn before I get caught? [Reddit]

.
* Based on unique clicks.
** Please take a look at last week's issue #653 here.

Cutting Room Floor

• Expertise in the Age of AI

• We should be more tired than the model

• Domain Expertise Has Always Been the Real Moat

• SQLite is All You Need for Durable Workflows

• Implications of the Draft FDA Bayesian Guidance

.

Whenever you're ready, 3 ways we can help:

1. Go deeper each week (paid subscription)
   Get 3 additional posts per week designed to help you:

   • Statistics → understand the math behind ML

   • AI Agents → build with modern AI tools

   • Career → become more valuable at your job

   👉 Upgrade for $10/month — cancel anytime
   

2. Looking to get a job?
   A practical guide to landing your first (or next) data science role, based on thousands of reader questions.
   👉 Check out our “Get A Data Science Job” Course
   

3. Promote your organization/project/event to ~68,500 subscribers
   Sponsor this newsletter and reach a highly engaged data science audience (30–35% open rate).
   👉 Reply to this email to learn more

Thank you for joining us this week! :)

Stay Data Science-y!

All our best,
Hannah & Sebastian

Issue #653
May 28, 2026

Hello!

Once a week, we write this email to share the links we thought were worth sharing in the Data Science, ML, AI, Data Visualization, and ML/Data Engineering worlds.

And now…let’s dive into some interesting links from this week.

Editor's Picks

• The quadratic sandwich
  If you have ever tried to minimize a function with gradient descent, you probably noticed that some functions are a joy to optimize and others are a nightmare. The difference often boils down to two properties: strong convexity and L-smoothness. These two concepts define a “sandwich” of quadratic bounds around your function that tells you exactly how well-behaved it is. If the sandwich is tight, life is good. If one slice of bread is missing, things get ugly fast…In this post we’ll build up both concepts from scratch, see how they combine into the quadratic sandwich, understand what happens at the level of the Hessian’s eigenvalues, and pick up a neat trick to verify L-smoothness without ever computing an eigenvalue…

• What it takes to transpose a matrix
  In this article we are going to gradually build a sequence of progressively more efficient implementations of matrix transpose, with the most sophisticated implementation being up to x25 times faster than the naive one. During each step we will locate the bottleneck, figure out what has caused it, and think of a solution to overcome it. This article is intended to serve as an introduction to optimizing matrix algorithms for x86_64, presented from the perspective of a real-world problem…

• Why is Kullback-Leibler divergence not a distance?
  The Kullback-Leibler divergence between two probability distributions is a measure of how different the two distributions are. It is sometimes called a distance, but it’s not a distance in the usual sense because it’s not symmetric. At first this asymmetry may seem like a bug, but it’s a feature. We’ll explain why it’s useful to measure the difference between two probability distributions in an asymmetric way. The Kullback-Leibler divergence between two random variables X and Y is defined as…

What’s on your mind

This Week’s Poll:

.

Last Week’s Poll:

.

Data Science Articles & Videos

• Friday Pins vs Sunday Pins or: How to Illustrate Something Completely Obvious
  In my previous article, I Spent the Last Month and a Half Building a Model that Visualizes Strategic Golf, I laid out the very basics of the golf model I built, the underlying reason that compelled me to work on it, and novel maps it could create. However, I barely scratched the surface of what this model can illustrate about golf course architecture. Here, I want to talk about how we can specifically look at one kind of dynamic architectural interest. That is, features of golf architecture that appear as we change the course setup. Specifically, I want to look at how different hole locations change the architectural interest for players, how we can show that, and what it looks like…
  

• What DS job market trends are you seeing? [Reddit]

  I have 20 YOE, but I do a generic “data science” search on LinkedIn every 3 months to see how the job market is trending. Here are my latest observations. I would love to hear what others think.

  1. The number of AI postings is going down. ML and DE skills are back in fashion.

  2. Salaries are down across the board.

  3. Non-technical responsibility is up. I see “Data Scientist” roles being asked to create a roadmap and drive organizational change. That used to be the responsibility of the manager or maybe the lead.

  I haven’t applied for any of these jobs, so I don’t know what’s actually real. I wonder if Data Science is no longer the hot keyword and I should be searching for something else…

• Thoughts About the Roles of AI for Statistics
  This talk covers what I’ve learned from using large language models in my work for the past two years. For statistical programming, success has come when I play the role of specification writer and comprehensive tester. For statistical methodology, AI has been successful serving me as a mathematical statistical assistant and a critic. Instead of avoiding AI we should embrace it, but we should always set a higher bar for the quality of our work as a result…
  

• Evaluating detection & classifier algorithm accuracy
  Let’s say i have images with a mixture of normal cells and sick cells on each image. Humans can reliably distinguish normal cells from sick cells, however it takes a lot of time to mark up the images as there are hundreds of cells in one field of view. I have an algorithm that can also distinguish normal and sick cells. The outputs from both manual markup and my algorithm is 2 lists of (x, y) coordinates -- one list for sick cells, one for healthy. What are the best practices for comparing and reporting the accuracy of my algorithm against manual markup?…
  

• Do Most TV Shows Stick the Landing?

  Four decades later, television has changed dramatically, reshaped by streaming and a clearer understanding of what makes for a satisfying conclusion. But has this institutional knowledge led to better endings? Have showrunners learned from the mistakes of St. Elsewhere, Game of Thrones, and other finale fiascos? So today, we’ll investigate whether omniscient showrunner Tommy Westphall has gotten any better at sticking the landing, how finale quality has changed in recent decades, and whether finality is simply a structural weakness of television itself…
  

• NYED Data Explorer Shows 15 Years of Charter School Success
  When I discovered 15 years of NYED assessment data, the interest to clean and free this data for others to discover in a Shiny app, was immediate. The opportunity to also feature Classical’s stand-out performance didn’t hurt my motivation, although this post and the app were built in my spare time, and do not represent the opinions of South Bronx Classical Charter Schools. Unlike many past Redwall posts, this one will not have code, and will be primarily to explain the data and show how to use the app…


• Luck vs. skill in poker
  The thread of our recent discussion of quantifying luck vs. skill in sports turned to poker, motivating the present post…Can good poker players really “read” my cards and figure out what’s in my hand?…
  

• Speeding up Stan model builds for R package developers
  My PhD student was interested in Bayesian methods and we put together an R package which included some Stan models. I was always frustrated by how slowly these compiled on our Windows machines…A few years later, when I got a MacBook Air I was shocked how much faster they compiled. On my Windows machine our mrbayes package takes 3 minutes 55 seconds to compile and install. On my M4 MacBook Air it takes 1 minute 16 seconds. The following tips show how to improve those timings…
  

• How unfair is the coin?

  In February 2024, Reverie Labs, the startup I co-founded in 2017, was acquired by Ginkgo Bioworks. I’m now on leave from Ginkgo and I’ve joined Y Combinator as a Visiting Partner, giving me the chance to work with the next generation of companies. Especially in this new role, I’ve been thinking a bit about what worked, what didn’t work, and what lessons I can take forward…We had quite the journey – 6+ years of building at the intersection of AI and drug discovery. We began as a machine learning driven software company selling SaaS tools and consulting services to pharma companies, and at acquisition we were a pharmaceutical company, developing our own in-house pipeline of drug assets and advancing them rapidly using our machine learning technology…
  

• Functions over Idioms - Writing R in Python with rfuns
  Sometimes a problem calls for a particular language to be used, and with that comes adjusting one’s brain to thinking in that language and using the appropriate idioms to leverage that language’s features…But what if I don’t want to?…The line between R and Python has been heavily blurred the last few years, particularly with {reticulate} (rstudio.github.io) enabling us to use Python within R code, RStudio rebranding as Posit (posit.co) and taking on a strong Python development effort, releasing Positron (posit.co) as a multi-language IDE, and Quarto (quarto.org) being a multi-language rethink of Rmarkdown…
  

• LLM Prompting Techniques for Data Scientists and Engineers in 2026

  Six techniques matched to six failure modes, including inconsistent output formats, shallow reasoning, instruction drift, and more…
  

• I am faking my way through a Data Analyst role with AI, how do I actually learn before I get caught? [Reddit]
  I graduated with a CS degree, but I spent my undergrad years grinding part-time jobs instead of actually studying. Now I am a Data Analyst at a small business, and the job is nothing like the theory I slept through in school. I am just winging it every day tbh. I rely heavily on openclaw for data scraping and acciowork to handle the processing and archiving. If these AI tools ever went down, I would be fired within an hour. I am terrified of being exposed as a fraud. Where do I even start fixing this? Should I grind python, or is mastering excel still the first step for survival?…
  

• 50 Hours to Draw Some Lines
  "What are you working on these days?"
  "Data visualizations." I told him.
  "Ah, you using algorithms, machine learning, cloud computing, things like that?"
  "No." I said. "I'm just trying to draw a line graph."….What do I mean by drawing data by hand? I made this data visualization (data viz) about a coffee maker computer by hand, using rulers, pencils, ink, and a lettering kit. Along with my flubs, flukes, and acclimation with tools - it took me 50 hours to make. It’s statistically accurate, carefully crafted, and like Hackaday said “right out of a 1970’s college textbook”. It’s how professionals might visualize data before computers could do it for them….

.

Last Week's Newsletter's 3 Most Clicked Links

• Are there any small, quick things I can do everyday to keep my skills sharp? [Reddit]

• After 5 years in data science, I’m starting to realize most “insights” we deliver are completely ignored. Is this normal? [Reddit]

• Is logistic regression regression?

.
* Based on unique clicks.
** Please take a look at last week's issue #652 here.

Cutting Room Floor

• Lahman Baseball Database, created by SABR member Sean Lahman, contains complete major league batting and pitching statistics back to 1871

• A Data-Driven Survey of MLB Franchise Management

• Behavior-Driven Development in R Shiny: Modeling User Behavior with When Steps

• I dont understand Standard Deviation [Reddit]

• Good practices in data scripts [Reddit]

.

Whenever you're ready, 3 ways we can help:

1. Go deeper each week (paid subscription)
   Get 3 additional posts per week designed to help you:

   • Statistics → understand the math behind ML

   • AI Agents → build with modern AI tools

   • Career → become more valuable at your job

   👉 Upgrade for $10/month — cancel anytime
   

2. Looking to get a job?
   A practical guide to landing your first (or next) data science role, based on thousands of reader questions.
   👉 Check out our “Get A Data Science Job” Course
   

3. Promote your organization/project/event to ~68,500 subscribers
   Sponsor this newsletter and reach a highly engaged data science audience (30–35% open rate).
   👉 Reply to this email to learn more

Thank you for joining us this week! :)

Stay Data Science-y!

All our best,
Hannah & Sebastian

Read more

Issue #652
May 21, 2026

Hello!

Once a week, we write this email to share the links we thought were worth sharing in the Data Science, ML, AI, Data Visualization, and ML/Data Engineering worlds.

And now…let’s dive into some interesting links from this week.

Editor's Picks

• What’s going on in computational neuroscience nowadays? (part 1)
  A month ago I came back from Cosyne, the annual Computational and Systems Neuroscience conference…The days are a haze of tutorials, talks, poster sessions, and workshops, usually appended by dinners and drinks past midnight…I seem to have a hard time writing one-off pieces, so I’m leaning into that and writing this as a series. I’ll only be able to write a very narrow perspective of Cosyne, of course, but most of the talks are on YouTube on the official channel if you want to see for yourself. (That also means this will be more personal thoughts than report)…

• Is logistic regression regression?
  I came across a post recently by a machine learning engineer who made the bold claim that logistic regression is the worst name for an algorithm ever, or something along those lines…Many statisticians of the more old-school type seemed to disagree. This led me to think a bit more deeply about the subject. I’ve already written several posts on bad terminology in statistics (see confidence level, line of best fit, r squared) so I might have been expected to agree with the machine learning view, but in this case I agree with the statisticians, and I would like to explain why…

• What Every Experimenter Must Know About Randomization
  Randomized controlled experiments offer gold-standard insight into cause and effect. The knowledge that informs our most important decisions. Unfortunately, randomization in such experiments is often botched. Randomization errors silently invalidate the interpretation of experimental results, turning a fruitful quest for knowledge into a waste of time and money, or, worse, a wellspring of misinformation. Fortunately, these fatal errors are easy to spot and fix. So whether you’re a webmaster using A/B testing to increase engagement, a medical researcher evaluating vaccines, a factory manager exploring productivity improvements, or a scientist seeking the laws that govern nature or human affairs, read on…

What’s on your mind

This Week’s Poll:

.

Last Week’s Poll:

.

Data Science Articles & Videos

• Converting testthat Tests to testit
  Back in 2013, I wrote about testing R packages when I first released testit. Thirteen years later, I still believe that unit testing should be nothing more than “tell me if something unexpected happened.” Recently I converted a large testthat test suite to testit, and I thought I’d share a practical guide for anyone considering the same move…
  

• After 5 years in data science, I’m starting to realize most “insights” we deliver are completely ignored. Is this normal? [Reddit]

  I’ve been in data science roles (both analytics and ML) for about 5 years now across a couple of companies. Lately I’ve been feeling a bit burned out because I keep seeing the same pattern…We spend weeks cleaning data, building dashboards, running statistical analysis, or training models… and then the stakeholders either:

  • Say “thanks” and never use it

  • Cherry-pick the numbers that support their existing opinion

  • Or just completely ignore the findings and go with gut feel anyway

  The worst part is when leadership asks for a “data-driven decision” but they’ve already decided what they want to do…Am I alone in this? Or is this just the reality of data science in most companies?…

• Tagging my blog posts with BERTopic and LLMs
  I recently added tags to my blog using BERTopic and a mix of LLMs. You can see the tags in the sidebar to the right (or in the footer on mobile). I’ve done this before in 2023, with GGUF Mistral using llama-cpp, but never finished the project. Now, because the models have been getting so good, and my project was small, relatively well-defined, and easy to evaluate, the project took me about 6-10 hours over a month, using BERTopic, Claude Code, and Pi with Deepseek…
  

• What data science is actually about in the age of AI
  I reflect on the evolving role of data scientists in the age of AI and LLMs. I argue that our core mission remains rigorous measurement, not full-stack development. While AI tools make building easier, the real value comes from defining and evaluating what truly matters. I share why measurement should be led by those closest to the problem and how data scientists can best contribute. Are we losing sight of what makes data science essential in the rush to build with AI?…
  

• Transformer From Scratch

  I’ve wanted to dive deeper into the fundamentals of AI for a while now - it feels a little bit magical, and a little bit wrong, to operate alongside AI without a strong understanding of how the underlying mechanisms work. Naturally, I had to write a transformer, and Neel Nanda’s ”GPT-2 From Scratch” was my resource of choice…This post is meant to document my process of learning and to address some of the questions I was curious about when implementing the transformer for the first time. It includes an overview of transformer basics and some of my intuitions, followed by some of the points of interest (transformer secrets, if you will) and challenges I ran into…
  

• The last six months in LLMs in five minutes
  I presented this lightning talk at PyCon US 2026, attempting to summarize the last six months of developments in LLMs in five minutes….


• Five tips for managing your R-universe
  rOpenSci’s R-universe system is an open source platform allowing users to create their own CRAN-like universe of R packages…This post gives five tips I have developed to help manage my R-universe…
  

• Research-Driven Agents: What Happens When Your Agent Reads Before It Codes
  Coding agents working from code alone generate shallow hypotheses. Adding a research phase — arxiv papers, competing forks, other backends — produced 5 kernel fusions that made llama.cpp CPU inference 15% faster.
  

• Kalman and Extended Kalman Filters: Concept, Derivation and Properties

  This report presents and derives the Kalman filter and the Extended Kalman filter dynamics. The general filtering problem is formulated and it is shown that, under linearity and Gaussian conditions on the systems dynamics, the general filter particularizes to the Kalman filter. It is shown that the Kalman filter is a linear, discrete time, finite dimensional time-varying system that evaluates the state estimate that minimizes the mean-square error…
  

• How not to turn power on its head
  In giving some informal remarks about power at a seminar a couple of weeks ago, I proposed that the tendency to turn the notion of power on its head might be avoided by imagining we need to define a test’s error probabilities in terms of its power alone. We can refer to the power against the null hypothesis, rather than alluding to a type 1 error probability, for example…What do I mean by turning power on its head? I mean, at least here, supposing that a test provides poor evidence of discrepancies that the test has low power to detect…
  

• The Atlas-Learn Approach to the Manifold Hypothesis

  The 2025 paper by Robinett et al., ‘Atlas-based Manifold Representations for Interpretable Riemannian Machine Learning’, provides an algorithm for fitting a low dimensional manifold from a point cloud by means of a novel algorithm for approximating an atlas of charts. This post illustrates the Atlas-Learn method by reconstructing a sphere from a 3D point cloud of naive random samples and works through some checks on accuracy…
  

• Are there any small, quick things I can do everyday to keep my skills sharp? [Reddit]
  I’m sure everyone knows about the dilemma of AI at this point. We want to work faster but our skills are atrophying yada yada…as a junior data scientist, I feel like I barely had any skills to begin with. Now with my company forcing us to use AI, I feel like I’m not learning much. Now I’ve been doing leetcode, but I just don’t think it’s that applicable to my real job. I don’t have the bandwidth outside of work to do a project yet, since my company is working us to the bone. What are some quick habits/tools/websites/apps you recommend to keep your skills sharp?..
  

• The Measurement of Loudness
  I’m not a sound engineer of any sort, but I enjoy music and have been blessed with decent hearing acuity, so I tend to pay attention to noises going on around me. Now, I know what you’re thinking! Surely, a measurement nerd interested in sound would have bought a cheap SPL meter off the internet and this is what this post is about. And you’d be wrong! Hah! Because this post goes a bit further off the deep end because SPL and the dB(a) scales that we commonly associate with “sound volume” always confused me when I tried to understand them. Like with how measuring color is really difficult because it’s at the intersection of a physical measurement and human perception (see: How the heck does one measure color?), sound is just as messy because it’s again physical measurements (sound pressure) mediated by the human auditory system. To measure loudness, we’re going to have to go back a bit in history…

.

Last Week's Newsletter's 3 Most Clicked Links

• Softmax, can you really derive the Jacobian? And should you care?

• Teaching <Fundamentals of Machine Learning>

• Illusions of Understanding in the Sciences

.
* Based on unique clicks.
** Please take a look at last week's issue #651 here.

Cutting Room Floor

• the vexing Hausdorff measure

• Multivariate reconciliation for hierarchical time series

• Comments on: What Every Experimenter Must Know About Randomization

• Introducing Metrics SQL: A SQL-based semantic layer for humans and agents

• Collaborating between Bioconductor and R-universe on Development of Common Infrastructure

.

Whenever you're ready, 3 ways we can help:

1. Go deeper each week (paid subscription)
   Get 3 additional posts per week designed to help you:

   • Statistics → understand the math behind ML

   • AI Agents → build with modern AI tools

   • Career → become more valuable at your job

   👉 Upgrade for $10/month — cancel anytime
   

2. Looking to get a job?
   A practical guide to landing your first (or next) data science role, based on thousands of reader questions.
   👉 Check out our “Get A Data Science Job” Course
   

3. Promote your organization/project/event to ~68,500 subscribers
   Sponsor this newsletter and reach a highly engaged data science audience (30–35% open rate).
   👉 Reply to this email to learn more

Thank you for joining us this week! :)

Stay Data Science-y!

All our best,
Hannah & Sebastian

Read more

Read more

Read more

Issue #651
May 14, 2026

Hello!

Once a week, we write this email to share the links we thought were worth sharing in the Data Science, ML, AI, Data Visualization, and ML/Data Engineering worlds.

And now…let’s dive into some interesting links from this week.

Editor's Picks

• Marco Polo: Finding a friend with only distance and motion.
  You walk into a cafe, looking for your friend. Seems like an easy task, until you see it’s so packed that you can’t see through the crowd at all, and everyone’s talking so loud that you can barely hear anything. The only things you know are your movements, and how far you are from your friend (through the special psychic bond you two share). How will you find each other?...I wanted to solve the exact same problem, but with devices instead of people (so no psychic connection for me), existing in a space of hundreds of other devices. Working the problem taught me a lot of really interesting science relating to robotics and state estimation, and I wrote this post so you can learn, too…

• Biology is a Burrito
  A bacterium’s genome, pulled into a straight thread, is nearly 1,000 times longer than the cell from which it came. If you placed one E. coli into a gallon-sized jug with some nutrients and waited a few hours, the genomes of its descendants, placed end-to-end, would reach to the moon and back...several times….The truth is that biochemistry textbooks often depict cells as spacious places, where molecules float in secluded harmony. “But a cell looks more like a burrito,” says Michael Elowitz, a biologist at Caltech. All the biochemicals are pushed together, bumping into each other…

• Softmax, can you really derive the Jacobian? And should you care?
  Multiclass output? Softmax. Normalising probabilities? Softmax. Attention weights? Softmax. Partition function? You guessed it, Softmax. This function comes up everywhere, but how often have you really thought about what’s going on inside?…What does softmax actually do to your distribution?…The softmax function is deceptively simple…

What’s on your mind

This Week’s Poll:

.

Last Week’s Poll:

.

Data Science Articles & Videos

• Illusions of Understanding in the Sciences
  The first part of this essay supports the case for the universality of partial and incomplete levels of understanding by showing the difficulty of reaching a deep level of understanding for even a simple analysis and model that most scientists use and believe they understand: linear regression. The second part highlights some implications of the existence of many levels of understanding and explanation, and their use by scientists for design, testing, analysis, and theory development. It discusses the way that deduction and induction depend on the levels of understanding and the implications of the illusion that a scientist’s understanding is deep…
  

• People Interested in Continual Learning Research [Reddit]

  Recently, I’ve become fascinated by Continual Learning, especially the idea of AI systems that can continuously adapt and improve from experience rather than staying static after training. I’m a student just starting my journey in CL research and would love to connect with people exploring similar ideas. Whether you’re a student, researcher, or just curious about the field, feel free to DM me. Would also love paper recommendations and interesting research directions…

• You Should Probably Map That: Introduction to Geospatial Analysis in R
  Anjile An, Weill Cornell Medical College…Data comes in many forms, but the spatial components can be overlooked. You’ll get some history of mapping and learn the basic building blocks of spatial data. The demo portion will go over how to use the trusty {ggplot2} as well as some new tools like {sf} and {tmap} to plot different types of spatial data…
  

• Requirements analysis: catching requirement bugs before they become code
  Every experienced engineer has a story where a feature shipped, worked on the happy path, and then quietly did the wrong thing on some edge case no one had thought about. Trace the bug back far enough and it rarely ends at the code; it ends at a sentence in a requirement document that meant one thing to the person who wrote it and something else to the person who implemented it…
  

• Teaching <Fundamentals of Machine Learning>

  This past spring, i taught <Fundamentals of Machine Learning> for computer science seniors (with some juniors as well as seniors from other majors, including data science and economics) at NYU. last time i taught this course, the course was titled <Introduction to Machine Learning>, it was pre-ChatGPT and it was pre-pandemic; in fact, i was teaching this course in the spring of 2020, and the whole university, city and world went into its first lock down mid-way. in other words, i taught this course in the old world, and i was asked to teach this course in this brave new world…
  

• Interactive Jensen–Shannon Divergence Visualisation
  An interactive visualisation of Jensen–Shannon divergence - the symmetric, always-finite cousin of KL. Shape two distributions and watch JSD, its ceiling of one bit, and the per-point contribution respond in real time…


• 70 years of love, empowerment, and freedom. Here’s a look at Eurovision by its lyrics.
  Famous for its upbeat rhythms, flamboyant performances, and political voting patterns, the competition has another dimension that moves with the zeitgeist: its song lyrics…
  

• What I’m Hearing About Cognitive Debt (So Far)
  A week ago, I wrote about how Generative and Agentic AI may be amplifying what I’ve been calling cognitive debt: the accumulated gap between a system’s evolving structure and a team’s shared understanding of how and why that system works and can be changed over time. The post sparked thoughtful discussion across different communities. Rather than respond thread by thread, I want to synthesize what I’m hearing and connect it to other reflections I’ve been reading. I will likely update this as the conversation evolves…
  

• Polynomial autoencoder

  The most direct way to compress an embedding (other than quantization) is to fit PCA on the corpus and keep the top-d eigenvectors. It works, but PCA is a linear projection, and neural-network embeddings on the sphere are structurally nonlinear — the well-known cone effect in transformers. Some of the variance lives in a nonlinear tail that a linear decoder can’t reach…This post is about a closed-form way to add a quadratic decoder on top of PCA, to capture part of that nonlinear tail. The encoder stays as plain PCA. The decoder is a degree-2 polynomial lift plus Ridge OLS (ordinary linear regression with L2 regularization), also closed-form. No SGD, no epochs, no hyperparameter search. One np.linalg.solve over corpus statistics…
  

• Visualize the Brrr - Learn GPU programming
  Who This Is For: This is for anyone interested in GPU programming or performance engineering, whether you’re writing CUDA, HIP, Triton, cuTile, Gluon, Helion, JAX, or nothing at all. If you’ve ever struggled to conceptualize parallelism, memory coalescing, or tiling, these visualizations are meant to make those ideas concrete. No prior GPU experience required.

  What You’ll Learn: The lessons walk through fundamental concepts of GPU programming, parallelism, memory hierarchy, and more. Each chapter pairs a short explanation with an interactive visualization you can poke at…
  

• PyTrendy

  PyTrendy is a robust solution for identifying and analyzing trends in time series. Unlike other trend detection packages, it is robust to noisy & flat segments, and handles for gradual & abrupt trend cases with a high precision. It aims to be the best package for trend detection in python…
  

• Standardization vs Log transform? [Reddit]
  I have been trying to understand the use cases of both of these and I am really confused. I know log transform fixes the features and makes their distribution normal and standardization on the other hand only fixes the scale of the feature by keeping the distribution the same. Are these things which I use one after the other ? Or just simply use one depending on the case (which I also don’t understand when) ?…
  

• AI agents can create convincing ecological models, but you still need to know what you’re doing
  Agentic AI tools like Claude Code can write and run code, fix its own errors, and produce a formatted report with figures. I wanted to know whether that translates into reliable ecological modelling, so we ran a test: three fisheries tasks, four AI models, ten independent runs each, scored against a rubric. The results are published in Fish and Fisheries. We found agents can be genuinely useful, but only if you know how to use them well and only if you know enough about the analysis to catch what they miss…

.

Last Week's Newsletter's 3 Most Clicked Links

• A decade of being an average Data Scientist! My personal experience [Reddit].

• Notes from inside China’s AI labs: Lessons from my trip to talk to most of the leading AI labs in China.

• DS market is kind of insane right now [Reddit]

.
* Based on unique clicks.
** Please take a look at last week's issue #650 here.

Cutting Room Floor

• Notes on Category Theory with examples from basic mathematics

• MySQL for Developers

• Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity

• Design your plots (ggplot) for mobile

• Assessing Credit Score Prediction Reliability Using Bootstrap Resampling

• The Figure Problem in Scientific AI: Building a Multimodal Literature Agent for Biology, Powered by NVIDIA Nemotron 3 Nano Omni

• Publishing a Quarto Blog: What I Learned Moving from Netlify to GitHub Pages

.

Whenever you're ready, 3 ways we can help:

1. Go deeper each week (paid subscription)
   Get 3 additional posts per week designed to help you:

   • Statistics → understand the math behind ML

   • AI Agents → build with modern AI tools

   • Career → become more valuable at your job

   👉 Upgrade for $10/month — cancel anytime
   

2. Looking to get a job?
   A practical guide to landing your first (or next) data science role, based on thousands of reader questions.
   👉 Check out our “Get A Data Science Job” Course
   

3. Promote your organization/project/event to ~68,500 subscribers
   Sponsor this newsletter and reach a highly engaged data science audience (30–35% open rate).
   👉 Reply to this email to learn more

Thank you for joining us this week! :)

Stay Data Science-y!

All our best,
Hannah & Sebastian

Read more

Read more

Issue #650
May 07, 2026

Hello!

Once a week, we write this email to share the links we thought were worth sharing in the Data Science, ML, AI, Data Visualization, and ML/Data Engineering worlds.

And now…let’s dive into some interesting links from this week.

Editor's Picks

• Notes from inside China’s AI labs
  Lessons from my trip to talk to most of the leading AI labs in China…

• Learning & Exploring Survival Analysis Part 1 - A Note To Myself
  A note to myself on survival analysis — KM curves, log-rank tests & Cox models 🧮 If I wrote it the way I understood it, maybe I’ll actually remember it 🤞…

• Two Cartography Expert Review MOVIE MAPS
  John Nelson and Peter Attwood review maps in films. What they get right, what they might get wrong, and what we think of them as cartography nerds. Includes thoughts on: Indiana Jones, The Lord of the Rings, Avatar, War Games, Prometheus, Harry Potter, The Muppets, Pirates of the Caribbean, Game of Thrones, Moonrise Kingdom, The Goonies, and The Emperor’s New Groove…

What’s on your mind

This Week’s Poll:

.

Last Week’s Poll:

.

Data Science Articles & Videos

• Bad Weather and the Subway
  I’ve been looking at hourly ridership data from the New York City Subway. Last time we learned that people go to work in the morning and come home in the evening, for example. (All together now: “Only in New York, baby!”) Today, we’ll learn that bad weather makes people stay at home. Except, sometimes it doesn’t…
  

• A decade of being an average Data Scientist! My personal experience. [Reddit]

  Hello! I know there’s people here with PhDs, working in FAANG, on top of the newest tech, and are absolutely brilliant Data Scientists. I’m not one of them…I just wanted to share my positive experience from someone who is painfully average lol!! I wanted to show people, especially new grads and/or people pivoting into the field, that you don’t have to be the smartest person in the room to get hired. You need to drill into the solid foundations and a have a drive to make change/bring value to a company…

• The World Inside Neural Networks
  How neural geometry will unlock understanding and control of AI…If we understand how a model carves up and represents the world conceptually (i.e., its ontology), this will unlock a far deeper understanding of both its algorithms that operate over that ontology and the intelligent behaviors produced by those algorithms. We thus need new methods to gain that understanding; this series of posts details our early efforts to develop them, building upon – and alongside – numerous related efforts from others…
  

• Quantum Machine Learning: The Pragmatic Guide for classical ML Engineers
  Part 1 of the “Quantum ML for Engineers” series: From Transformers and GPUs to QPUs and Hybrid Intelligence…
  

• Bringing OpenTelemetry to R in production
  Posit has instrumented Shiny, plumber2, mirai, httr2, ellmer, knitr, testthat, and DBI with OpenTelemetry, and created tools for you to instrument your own packages, bringing production-grade observability to R…
  

• Data 100: Principles and Techniques of Data Science UC Berkeley, Spring 2026
  This intermediate level class bridges between Data 8 and upper division computer science and statistics courses as well as methods courses in other fields. In this class, we explore key areas of data science including question formulation, data collection and cleaning, visualization, statistical inference, predictive modeling, and decision making.​ Through a strong emphasis on data centric computing, quantitative critical thinking, and exploratory data analysis, this class covers key principles and techniques of data science…


• Inverse Probability Weighting: From Survey Sampling to Evidence Estimation
  We consider the class of inverse probability weight (IPW) estimators, including the popular Horvitz–Thompson and Hájek estimators used routinely in survey sampling, causal inference and for Bayesian computation. We focus on the ‘weak paradoxes’ for these estimators due to two counterexamples by Basu (1988) and Wasserman (2004) and investigate the two natural Bayesian answers to this problem: one based on binning and smoothing: a ‘Bayesian sieve’ and the other based on a conjugate hierarchical model that allows borrowing information via exchangeability…
  

• The Magic of In-Context Learning (ICL): When Your Model Already Knows Your Data
  As an experienced data scientist, you have seen thousands of datasets in your career. When confronted with new data, your natural neural network (a.k.a. brain) simply draws on this vast library of past mathematical shapes and immediately recognizes the pattern. But what if an artificial neural network could do exactly the same thing? What if it could predict your data without actually being trained on it?…Welcome to the mind-bending world of In-Context Learning (ICL) for tabular data, brought to R via the incredible new TabPFN package (on CRAN)…
  

• Three ways to differentiate ReLU

  When a function is not differentiable in the classical sense there are multiple ways to compute a generalized derivative. This post will look at three generalizations of the classical derivative, each applied to the ReLU (rectified linear unit) function. The ReLU function is a commonly used activation function for neural networks. It’s also called the ramp function for obvious reasons…
  

• You Don’t Need to Learn All the Weights on tabular data: The Case for rvflnet (a nonlinear expressive glmnet) on regression, classification and survival analysis
  Random Vector Functional Link (RVFL) networks offer a simple yet powerful alternative to traditional neural networks for tabular data. Instead of learning hidden layers through backpropagation, RVFL generates them randomly (or not, if using a deterministic sequence of quasi-random numbers) and focuses all learning effort on a final, regularized linear model…
  

• Comparing R’s {targets} and dbt for Data Engineering

  I’m getting more and more into data engineering these days and having used R for a long time, I’m seeing a lot of problems that look nail-shaped to my R-shaped hammer. The available tools to solve those problems exist for (presumably) very good reasons, so I wanted to take some time to dig into how to use them and compare their workflows to what I would otherwise naively do in R…
  

• DS market is kind of insane right now [Reddit]
  So here’s the story: another team in my company opened an associate-level DS role last week, we got 300+ applications, and somehow there were 30+ senior-level guys applying for it. Not fake senior either. Like actually senior all with 10+ yoe….Curious that are other people & teams seeing the same thing, or is this just a weird sample on our side?…
  

• Visualizing History: The Polish System
  For the Polish educator Antoni Jażwiński, history was best represented by an abstract grid — or at least it was for the purposes of remembering it. The so-called “Polish System” originated in the 1820s and was later brought to public attention in the 1830s and 1840s by General Józef Bem, a military engineer with a penchant for mnemonics…

.

Last Week's Newsletter's 3 Most Clicked Links

• Warning: Don’t get GPT-brained [Reddit]

• Principles for Autonomous System Design: OpenClaw Deep Dive

• How The Heck Does Shazam Work?

.
* Based on unique clicks.
** Please take a look at last week's issue #649 here.

Cutting Room Floor

• Planning for your midlife crisis (or Counterfactual Analysis with Bayesian Models: What Drives the Life Expectancy Gap?)

• My Thoughts on AI, Part 1: Fears, Opinions, and Mental Journey

• Reflections on AI-assisted Programming

• Time series forecasting with tabular foundation models

• Should You Trust the Netflix Top 10? A Statistical Analysis

.

Whenever you're ready, 3 ways we can help:

1. Go deeper each week (paid subscription)
   Get 3 additional posts per week designed to help you:

   • Statistics → understand the math behind ML

   • AI Agents → build with modern AI tools

   • Career → become more valuable at your job

   👉 Upgrade for $10/month — cancel anytime
   

2. Looking to get a job?
   A practical guide to landing your first (or next) data science role, based on thousands of reader questions.
   👉 Check out our “Get A Data Science Job” Course
   

3. Promote your organization/project/event to ~68,500 subscribers
   Sponsor this newsletter and reach a highly engaged data science audience (30–35% open rate).
   👉 Reply to this email to learn more

Thank you for joining us this week! :)

Stay Data Science-y!

All our best,
Hannah & Sebastian

Issue #649
April 30, 2026

Hello!

Once a week, we write this email to share the links we thought were worth sharing in the Data Science, ML, AI, Data Visualization, and ML/Data Engineering worlds.

And now…let’s dive into some interesting links from this week.

Editor's Picks

• The Difference Between “Replicable” and “Not replicable” is not Itself Scientifically Replicable
  Replication studies estimate the replicability rate of scientific results by aggregating binary verdicts of experiments. Exact replications are rarely attainable, so most replication sequences are non-exact. Experiments differ in ways that matter and do not share a single data-generating process. We formalize two statistical interpretations of non-exactness. In a shared latent rate (benchmark) model, experiments are exchangeable and depend on a common random replicability rate. In a conditionally independent rates (operational) model, each experiment has its own replicability rate drawn from a population distribution………The replication crisis, if there is one, cannot be established by the methods used to declare it….

• You can beat the binary search
  Binary search is a classic algorithm that efficiently locates a target value in a sorted array by repeatedly dividing the search interval in half…In C++, this is implemented by the std::binary_search function, which returns a boolean indicating whether the value is present…The popular Roaring Bitmap format uses arrays of 16-bit integers of size ranging from 1 to 4096. We sometimes have to check whether a value is present. We use a binary search…I wanted a faster approach. I had two insights…Virtually all processors today have data parallel instructions (sometimes called SIMD) that can check several values at once…The binary search checks one value at a time. However, recent processors can load and check more than one value at once…Thus, I created something I call the SIMD Quad algorithm. It is an efficient search algorithm for sorted arrays of 16-bit unsigned integers, combining a quaternary interpolation search with SIMD (Single Instruction, Multiple Data)…

• Principles for Autonomous System Design: OpenClaw Deep Dive
  In this talk, Alex Krentsel (UC Berkeley, NetSys Lab / Google Research) does a deep-dive into OpenClaw — a fully open-source autonomous AI agent system — and uses it as a lens to explore the emerging design principles behind truly autonomous agents. We’re in Phase 3 of the AI evolution: LLM + tool-use + dynamic tool discovery. The agents that exist today aren’t chatbots. They read your email, write code, schedule work, remember context across sessions, and spawn other agents. This talk breaks down exactly how that works….

What’s on your mind

This Week’s Poll:

.

Last Week’s Poll:

.

Data Science Articles & Videos

• Linear Regression vs. Deep Learning: A Simple Yet Effective Baseline for Human Body Measurement
  We propose a linear regression model for the estimation of human body measurements. The input to the model only consists of the information that a person can self-estimate, such as height and weight. We evaluate our model against the state-of-the-art approaches for body measurement from point clouds and images, demonstrate the comparable performance with the best methods, and even outperform several deep learning models on public datasets. The simplicity of the proposed regression model makes it perfectly suitable as a baseline in addition to the convenience for applications such as the virtual try-on. To improve the repeatability of the results of our baseline and the competing methods, we provide guidelines toward standardized body measurement estimation…
  

• Warning: Don’t get GPT-brained [Reddit]

  At my last role we had to move fast, so we relied on an LLM to help with a lot of the thinking and coding for us so we could focus on the business use case and managing meetings and stakeholders. The role was heavy on project management as well as development, research, and deployment so basically doing everything While I got good at scoping projects and managing them, my technical skills totally deteriorated in less than 1 year. It’s scary going back to problems I know I can solve and but have some brain fog when getting to the answer. If I could have gone slower, had more time to thinking about modeling/coding than I probably wouldn’t feel like this Don’t get GPT brained. You’ll have to crawl out of that pit eventually. Like technical debt but for your brain…

• There Will Be a Scientific Theory of Deep Learning
  In this paper, we make the case that a scientific theory of deep learning is emerging. By this we mean a theory which characterizes important properties and statistics of the training process, hidden representations, final weights, and performance of neural networks. We pull together major strands of ongoing research in deep learning theory and identify five growing bodies of work that point toward such a theory: (a) solvable idealized settings that provide intuition for learning dynamics in realistic systems; (b) tractable limits that reveal insights into fundamental learning phenomena; (c) simple mathematical laws that capture important macroscopic observables; (d) theories of hyperparameters that disentangle them from the rest of the training process, leaving simpler systems behind; and (e) universal behaviors shared across systems and settings which clarify which phenomena call for explanation…
  

• What do the fundamental constants of physics tell us about life?
  In the 1970s, the renowned physicist Victor Weisskopf famously developed a research program to qualitatively explain properties of matter in terms of the fundamental constants of physics. But there was one type of matter prominently missing from Weisskopf’s analysis: life. Here, we develop Weisskopf-style arguments demonstrating how the fundamental constants of physics can be used to understand the properties of living systems. By combining biophysical arguments and dimensional analysis, we show that vital properties of chemical self-replicators, such as growth yield, minimum doubling time, and minimum power consumption in dormancy, can be quantitatively estimated using fundamental physical constants. The calculations highlight how the laws of physics constrain chemistry-based life on Earth, and if it exists, elsewhere in our universe…
  

• Gaussian Process Regression (GPR) & Physics-driven Computing
  Gaussian Process Regression (GPR) is a non-parametric, Bayesian approach to regression. Unlike parametric models (like linear regression) where we find a distribution over parameters, a Gaussian Process defines a prior distribution over functions…
  

• Open Data Structures - An open content textbook
  Open Data Structures covers the implementation and analysis of data structures for sequences (lists), queues, priority queues, unordered dictionaries, ordered dictionaries, and graphs…Data structures presented in the book include stacks, queues, deques, and lists implemented as arrays and linked-lists; space-efficient implementations of lists; skip lists; hash tables and hash codes; binary search trees including treaps, scapegoat trees, and red-black trees; integer searching structures including binary tries, x-fast tries, and y-fast tries; heaps, including implicit binary heaps and randomized meldable heaps; graphs, including adjacency matrix and ajacency list representations; and B-trees…


• PostgreSQL is Not Slow. Your Queries Are.
  A field guide to the seven things that are actually making our database feel slow and how to stop blaming the wrong suspect…Culprit #1: The Missing Index…Culprit #2: The N+1 Query, Death by a Thousand Cuts…Culprit #3: Stale Statistics, The Planner Is Flying Blind…Culprit #4: The Query That Runs Fine, Alone…Culprit #5: EXPLAIN ANALYZE Exists, Use It…Culprit #6: Connection Exhaustion, PostgreSQL is Full…Culprit #7: Reporting Queries Running on Production…
  

• An interactive semantic map of the latest 10 million published papers
  I built a map to help navigate the complex scientific landscape through spatial exploration. How it works: Sourced the latest 10M papers from OpenAlex and generated embeddings using SPECTER 2 on titles and abstracts. Reduced dimensionality with UMAP, then applied Voronoi partitioning on density peaks to create distinct semantic neighborhoods. The floating topic labels are generated via custom labelling algorithms (definitely still a work in progress!). There is also support for both keyword and semantic queries, and there’s an analytics layer for ranking institutions, authors, and topics etc…
  

• How The Heck Does Shazam Work?

  How audio fingerprinting and a connect-the-dots trick lets Shazam identify a song in seconds…By throwing away almost everything and keeping only a handful of landmark peaks, a noisy 5-second clip from a coffee shop becomes a set of coordinates precise enough to pinpoint one song out of millions. Recognition, it turns out, is mostly an exercise in ignoring the right things…
  

• Generating Correlated Random Numbers in R Using Matrix Methods
  Generating random data with a specific correlation structure is a common need in statistical simulation. In this post, let’s walk through how to do it in R using matrix decomposition methods…
  

• Schotter Plots in R

  Translating things between languages reveals how each language approaches different design trade-offs, and I believe it’s a useful exercise. Having something to translate is the first step. I found a plot I wanted to generate (Georg Nees’ “Schotter” computer-generated art from 1968 which shows a grid of squares which get increasingly displaced in position and rotation), and some code that reproduced it, so off we go!…
  

• Why Most Time Series Models Fail Before They Start
  A practical look at stationarity and transformations with real CPI data in R…Many time series models fail before they even begin. Not because the software crashes. Not because the code is wrong. But because the data entering the model violate one of the most important assumptions in time series analysis: stationarity…The goal is simple: show why raw time series levels often mislead us, what stationarity really means, and why transformations such as differencing and log-differencing are not cosmetic tricks but conceptual necessities…
  

• Hourly Subway Station Flows
  Pie charts are bad, as any fule kno. We’re not as good at judging relative differences between angles and areas as we are at judging relative differences in lengths on a common baseline. This is especially true when we have more than two things to compare at the same time. So, as a rule, you shouldn’t use them. You should figure out some other way of viewing your data instead. On the other hand, I just made 424 animated pie charts because if you’re going to break a rule you should break it good and hard…

.

Last Week's Newsletter's 3 Most Clicked Links

• Your Name in Landsat 🛰️

• CEO cancels BI tooling, replaces it with AI, breaks everything [Reddit]

• Anyone else paranoid using AI for analysis? [Reddit]

.
* Based on unique clicks.
** Please take a look at last week's issue #648 here.

Cutting Room Floor

• How The Heck Does GPS Work?

• Does Postgres Scale?

• How do you test AI agents in production? The unpredictability is overwhelming. [Reddit]

• Do Loyalty Programs Actually Create Loyalty?

• logrittr: A Verbose Pipe Operator for Logging dplyr Pipelines

• A Data-Driven Survey of MLB Franchise Management - A search for factors leading to playoff success

.

Whenever you're ready, 3 ways we can help:

1. Go deeper each week (paid subscription)
   Get 3 additional posts per week designed to help you:

   • Statistics → understand the math behind ML

   • AI Agents → build with modern AI tools

   • Career → become more valuable at your job

   👉 Upgrade for $10/month — cancel anytime
   

2. Looking to get a job?
   A practical guide to landing your first (or next) data science role, based on thousands of reader questions.
   👉 Check out our “Get A Data Science Job” Course
   

3. Promote your organization/project/event to ~68,500 subscribers
   Sponsor this newsletter and reach a highly engaged data science audience (30–35% open rate).
   👉 Reply to this email to learn more

Thank you for joining us this week! :)

Stay Data Science-y!

All our best,
Hannah & Sebastian