It's not "common". You have to deal with StopIteration only when you write an iterator with the low-level API, which is maybe once in the career time for most of developers.
When I was incubating the idea, I thought about different concepts:
1. The current bring-your-own-key.
2. A central summary storage, filled by me.
3. A central summary storage, crowdsourced.
4. A paid subscription, where I effectively run some LLM proxy.
I wanted something low overhead and be just the right size for yet another weekend project which I could drop at any moment. Supporting some infrastructure, having moderation headaches, let alone receiving payments ruled out pretty much everything but the current approach.
That's understandable, I feel same when I install extensions.
In both browsers, you can install the extension from local disk instead of the browser stores. The release artifact is a ZIP file with plain JS inside, no bundling, minification, preprocessing, you can check it out.
Both Chrome and Mozilla did some inspection during several business days, but I can't say exactly what they checked and how diligently.
There is also (or at least used to be?) an OpenAI compatible API layer for Ollama so that may be an option as well, though my understanding is there are some downsides to using that.
Note: This comment and the link are just meant as references/conveniences, not intended as a request for free labor. Thanks for opening up the code!
Forget ollama, just changing the URL from openai to your local server is enough, llama.cpp has a compatible endpoint. Most people just don't bother giving the option since you get a CORS error if it doesn't have a valid cert.
1. After the recent successful growth of Antithesis [1], I'm diving into the topic of deterministic simulation testing. There are some cases (and ready-to-use libraries) where people are doing this in Rust, C++, Go, I'm interesting in this in Java. So I'm up to some experiments. I've also started an "awesome list" of resources about this topic [2]
2. I've made a generated serialization/deserialization library for the Kafka wire protocol in Rust, tested against the original Java implementation. I'll add 3.9 support once it's released and don't see much upcoming changes to the library, apart from maybe working on the Go version.
I've been working on a tool called PyHeap that allows making a running Python program (on CPython interpreter) dump its heap content on disk and later analyze and vizualize these files. This works without any modification to the program source code, CPython code, or even restart.
PyHeap uses GDB to attach to the target CPython process and inject a piece of Python code that writes the heap dump. The dump will not 100% complete, because the dumper relies only on what it can find in the run time using the standard modules `sys` and `gc` without support from CPython. For example, various native extensions like NumPy must be explicitly supported (not at the moment). However, it's good enough for many use cases.
GDB doesn't need to be installed "near" the target, only "near" the dumper itself. It mean, it will work even with e.g. dockerized Python apps without modification. Linux namespaces are generally supported.
A word on motivation. Java has nice tooling for saving (dumping) the content of the process' heap into a file. A running JVM or a JVM failing with out-of-memory (at least, HotSpot) can do a dump in a portable format. There are standard CLI (jmap, jcmd) and GUI (VisualVM) tools to initiate this [1]. Tools like Eclipse Memory Analyzer [2] or VisualVM [3] can read and query (OQL) these files. Literally see the values of fields, local variables, thread states, etc. This is a super powerful debugging technique. In my career I've figured out many tricky bugs by poring over a heap dump. I miss this debugging experience badly in the Python world. I decided to fill the gap and provide something similar to jmap + Eclipse Memory Analyzer or VisualVM heap dump viewer.
