Tag Archives: data science

App-ready databases from Python with SQLModel and FastAPI

Python is ubiquitous for those of us coming into programming from academic research backgrounds, both for generating scientific data and for analysing it. Of course, modern data-driven research only thrives when data is FAIR (findable, accessible, interoperable, and reusable) and traditional relational databases and APIs for interacting with them are a tried-and-true method of ensuring FAIRness. (And in the age of machine learning, the machines need things to be FAIR arguably even more than we do…)

“But I’m a data scientist/computational chemist/bioinformatician!” you protest. “I just (ab)use databases using Python, I don’t make them for someone else to use!” Understandable, but the barrier to entry may be much lower than you think, thanks to two related Python libraries: SQLModel and FastAPI. These powerful tools by developer Sebastián Ramírez (@tiangolo on GitHub) work in concert to make relational databases and APIs much easier to implement for Python natives. Let’s see how they can help…

Continue reading

Finding 250GB of Missing Storage On My Mac: A Warning For Large Dataset Users

I recently faced a puzzling issue: my 1TB MacBook Pro showed only 150GB free, but disk analyzers could only account for about 500GB of used space. After hours of troubleshooting, I discovered that Spotlight’s search index had balooned to 233GB, hundreds of times larger than normal.

The Problem

Standard disk analyzers showed that my mac had 330GB of “Inaccessible Disk Space” and 66GB of “Purgeable Disk Space” but no clear explanation for where my storage went. Removing the purgeable space was easy enough with sudo purge but none of the recommended fixes from ChatGPT like clearing Time Machine snapshots, clearing unused conda packages with pip cache purge and conda clean --all, and restarting the computer had any effect on the inaccessible disk space.

Continue reading