Posts

Overview This page will consist of two sections: a brief section on some lessons learned from this internship and a longer report describing in detail the agent architecture we used. For a detailed report on how our Django backend works, please see this. Feel free to reach out on LinkedIn or email me at jc3579@cornell.edu if you have questions. Lessons Learned, Part 2 In my first lessons learned post, I discussed how we used RAG in a specific way to address a specific use case: customizing advice based on referring doctor. This allowed us to have a different corpus of experts on whom to RAG based on which doctor was referring. However, this led us to one big issue: some more prolific experts, with more content on the internet, were overpowering the retrieval, even in areas not under their expertise. For example, if I asked a question about sleep, due to the sheer amount of his content, Mark Hyman’s content would always come out on top over Matthew Walker, the actual sleep expert. ...

Back in May, when first started working on one of my projects which uses PostgreSQL, I implemented a TypeScript edge function called delete-user that does exactly what you’d expect it to do (among other things), and it worked fine back then. But two days ago, to my dismay, I tried deleting an account, and I received a 500 error from the API call. After adding some logs, I narrowed the error down to this line: ...

Let’s say you have an app where users can search up any book. From the user side, it might seem like we need a database row for every single edition, with title, publisher, description, cover image, and so on. Especially for descriptions and cover images, when considering up to 50 million editions, costs can accrue quickly, not to mention acquiring such a database would be difficult. A solution which I am quite partial to is what is known in system design as lazy caching. What this means is that we only load something in the first time it is needed and we don’t have it, and for subsequent requests, since we already have it, we don’t need to load it again. What I find elegant about it is that for the cost of one user having slightly higher latency, database size is much smaller, it allows the database to be built over time rather than all at once, and most importantly, to the user it appears as if the whole database is present. ...

Two weeks ago, I started working at a startup called Jarvis In Your Pocket. Imagine if you could track your nutritional (and overall) health using just an LLM, an LLM that could learn about you, be proactive, and give solid advice based on trusted expert sources selected by your doctor. Some things were easier than I expected. First, we envisioned a virtual retrieval-augmented generation (RAG) system whereby we could consult with multiple doctors who had their own trusted materials and RAG on a specific subset of all the sources based on which doctor had referred the user. While it may sound like we have to create virtual vector databases on top of the large corpus of content, it’s actually as simple filtering on chunk metadata like so: ...