proxy in under 15 lines, you can inject your credentials as env vars in a seperate container:

client:

url = PROXY_URL api_key = TARGET_URL

proxy:

app = FastAPI()

@app.post("/{path:path}")

async def proxy(request: Request, path: str):

    target_url = request.headers.get("Authorization", "")[7:]

    api_key = os.environ.get("API_KEY")

    forward_headers = {"X-Subscription-Token": api_key, "Content-Type": "application/json"}

    r = await httpx.AsyncClient(timeout=600).post(target_url, headers=forward_headers, content=await request.body())

    return Response(content=r.content, status_code=r.status_code, headers=dict(r.headers))

#Telemetry FUCK OFF export DOTNET_CLI_TELEMETRY_OPTOUT=1 export ASTRO_TELEMETRY_DISABLED=1 export GATSBY_TELEMETRY_DISABLED=1 export HOMEBREW_NO_ANALYTICS=1 export NEXT_TELEMETRY_DISABLED=1 export DISABLE_ZAPIER_ANALYTICS=1 export TELEMETRY_DISABLED=1 export GH_TELEMETRY=false

Also:

  # Atlas
  export DISABLE_TELEMETRY=1
  # CloudFlare
  export WRANGLER_SEND_METRICS=false
  export VERCEL_PLUGIN_TELEMETRY=off
  # AWS
  export SAM_CLI_TELEMETRY=0
  export CDK_DISABLE_CLI_TELEMETRY=true
  # ???
  export DO_NOT_TRACK=true

Also.. Easily opt-out from telemetry collection https://github.com/beatcracker/toptout (last commit 3ya)

you can use a safety model trained on prompt injections with developer message priority.

user message becomes close to untrusted compared to dev prompt.

also post train it only outputs things like safe/unsafe so you are relatively deterministic on injection or no injection.

ie llama prompt guard, oss 120 safeguard.

Unfortunately it's not that simple. Self-policing AI systems will always be gamed. Just one [0] example of this among many.

[0] https://www.hiddenlayer.com/research/same-model-different-ha...

when ai is dead we can use all those gpus for zucc's metaverse xD s

there is no way to cap your billing on gcp.

you can get notifications but that's it.

i don't want to get throttled below my quota but some type of spend limit would be good.

and the notifications can be delayed because the spending system is not updated in real-time, so even if you have a Cloud Task triggering on spending to disable the project it may be too slow and several thousands may already be spent.

Is there a cloud provider that does have hard unbreakable billing caps? Everything I've seen has always been notifications or soft caps.

Not talking about fixed-access things like a Hetzer box.

Bunny.net purports to have a pay-as-you-go prepaid credit system that sounds like it works the way people want, and with their description of the way it works probably being sufficient to be legally enforceable if it turns out that it actually works differently and you were to end up with a surprise bill from them. And evidently it really does work that way; see this post from a couple weeks ago: <https://news.ycombinator.com/item?id=47676416>

The only other provider known to work that way is NearlyFreeSpeech.NET, which serves a completely different market segment (so much so that it might as well not even be considered the same kind of product/service).

Prepaid only is a fantastic idea, especially for dumb-ass startups. Limiting your liability to $100 or so sound like a big-ass W.

Yes, pre-paid would be fine and it's a well-understood pattern.

No need to retire API keys.

Nobody is thinking about the stock owners, I see

> Of course it doesn't have to be perfect, but even approximating perfection doesn't look much different.

It's pretty easy to get right, if the provider allows you to go (slightly) negative before cutting you off.

> Also, can you imagine the kind of downtimes and complaints that would inevitably originate from a fully synchronous billing architecture?

Doesn't need to fully synchronous.

Open ai has this

Prepaid only is a fantastic idea, until your site goes (desirably) viral and then gets shut off right as traffic is picking up, or you grow steadily and forget to increase your deposit amount and suddenly production is down. Billing alerts are a much better solution IMHO.

No you big dummy, that is especially when you want to limit your liability, lol.

Because these days it will be all worthless bot traffic.

Prepaid/paid limits with shutoff is appropriate for this though.

If you have per key limits, this is not possible, and even in a wild situation you should b able to expect that your firebase key will not use 50k.

Let me choose. This common point seems more like a rationalization for the default behavior of hyperscalers. AWS isn't avoiding prepaid due to concern about my site's virality, just that prepaid = less money.

You can also have both, a cap and one or more billing alert levels below it. Some providers do this (e.g. IIRC Backblaze B2).

Yes in reality, and ideally, you can have both, but GP specifically said "Prepaid only" implying you can't have both (which is what I replied to)

Well, they should also have pre-paid only. Offer a few different options.

Oh please no. And the "alternatives" to API keys aren't going to help much either, they'll just add friction to getting started (as reference: see the pain involved in writing a script that hits gmail or calendar API)

cloudflare

i think at this point i have had enough of the majority of consumer products and just use production.

backup to real s3 storage.

llms on real api tokens.

search on real search api no adverts.

google account on workspace and gcp, no selling the data.

etc.

only way to stop corpos treating you like a doormat

you know once, anthropic was supposed to be a public benefit org!

tyrants and crooks can survive in the moment but can rarely hold a general rebellion. they can either flee or get pulled apart by the mob.

boots eventually tread the marble floors and empty halls. the new reich chancellery, the people's palace of syria, al faw, to name a few.

"You may hand us over to the executioner, but in three months time, the disgusted and harried people will bring you to book and drag you alive through the dirt in the streets." - General Erwin von Witzleben

it applies less to the oligarchy, all the same the good times probably do not roll on forever for regime cronies