2024-06-06

• Continued reading Brendan Gregg’s Systems Performance book. A thing that keeps coming up in various examples is optimizing I/O sizes. Basically, applications can have significant speedups by tuning I/O sizes so that you end up with a smaller number of I/O calls with larger size vs many I/O calls with small amounts of data. Reminded me of a data structure I learned about at a Dist Sys meetup in New York: Bε-trees. Basically, this B-tree variant is write-optimized, and the main way that is achieved is by collecting writes into an in-memory buffer before sending them to disk. This amortizes the cost of disk I/O over more inserts compared with traditional B-tree (and similar to LSM trees), which increases the write performance on average. This kind of I/O optimization also reminds me of the part in the GFS paper where they mentioned they use large OS page sizes to saturate disk bandwidth when reading large files. This dramatically increases throughput on their typical tasks (processing huge amounts of data across many machines).
  
• I ran another check on using vision-based GPT APIs to add useful info to PadDaddy (specifically, estimating natural light levels from photos of the unit). Unfortunately, it’s still too expensive for my budget to process all the incoming images using these APIs. There’s something like 5-10k new apartment images per day, each image uses ~600 tokens to produce an estimate (almost entirely input tokens), thus the cost is ~$20/day to process every image for every listing. Adding $600 to my monthly burn for PadDaddy isn’t something I’m going to do rn. If I could get it to $1/day, that’d be worthwhile. Of course, I could look at ways to provide the feature without processing every image. I could cut 60% of costs by only running the “Great” and “Good” listings through the analysis. But I need to cut costs by 95%, so… The dream is, ofc, to build systems where users pay for the tokens to do the processing they care about. That might be a good startup (users buy tokens in bulk, are able to spend them at any service that supports the api). Though, I would still expect users to prefer known, upfront costs, even if those costs on average are exorbitant compared to usage. The act of thinking “is this worth the tokens” for every type of processing is mental overhead that is annoying for customers and likely bad for business.
  
• So glad that Love Island is back. 3 episodes in, 57 to go. Group is lookin good already. Good chemistry. Good drama. Also teaching me a lot about the female experience.

---

---

---