Dran

I would argue that Linux kernel scale basically doesn’t work. It is a huge barrier to would-be developers to get into. You might argue that’s a feature, but it’s pretty hard to argue that it doesn’t add a lot of friction to the process

I’m glad I’m not the only one who is confused

Genuinely, pull requests being off platform without good integrated diff and merge conflict resolution tooling just doesn’t work at a certain scale.

I have no knowledge about the economics of live poultry, but rotisseries are generally loss leaders at grocery stores. They’re so commonly loss leaders that they’re often cited as one of the primary examples to explain the concept of a loss leader.

https://thehustle.co/the-economics-of-costco-rotisserie-chicken

https://en.wikipedia.org/wiki/Loss_leader

Is this hopium or did they release a statement suggesting an 3.8 35b a3b was in development for release?

I’m not aware of any public frontier LLM provider that uses a static seed for inference. Meaning, even with an identical prompt and identical model you will not get the same output. Seeds should absolutely come back with the streaming metadata on requests imho, but they don’t in any api/harness I’m aware of.

my guess is something like this

https://330k.github.io/misc_tools/unicode_steganography.html

A model could hypothetically be trained to insert zero-width characters (I doubt any have though). But any other layer could also very trivially insert these codes. The inference engine could be designed to delay output streaming by however many tokens is required to embed their coding and ninja-insert them during the decode stream. A proxy between the inference engine could insert them. A harness could insert them. Hell, even the rendering javascript frontend in your browser could insert them.

Either the inference engine or proxy would be the prime target if they want to enable this on api responses as well as copy/paste from a chat interface. They could also do a combination of the above depending on final output mode.

You are definitely correct though that it’d be trivial to detect and strip by someone aware of it.

Not necessarily, there are valid “characters” that are not rendered in most text parsers, or are rendered as whitespace. A great example of this is the byte-order-mark (BOM). You can embed a BOM code in the whitespaces of a text string and it looks exactly the same to a human as one without.

I assume that’s what they’re talking about here

Here’s an example of a steganography encoding technique using this method. https://330k.github.io/misc_tools/unicode_steganography.html

You can’t just write off capital expenditure though. The hardware, even for “effecient” MOE inference is still very expensive to buy, house, run, and cool. Even assuming open-weight model serving at $0 r&d for the models themselves, mixing high-prefill workloads doesn’t batch well with decode heavy concurrency (or other prefill-heavy jobs). The moment you do anything nontrivial you start running into very complicated architectural problems to efficiently solve at scale.

Hardware that is useful for 5-10 years at most, plus development and support for the inference workflows, doesn’t leave a lot of margin on the table.

My gut, along with basically everything I read, suggests that not most (even pure inference) shops are not profitable and are still floating on loans or vc money.

Can you cite your source on the claim that “inference is currently insanely profitable”? Everything I read suggests that openai and anthropic lose money on their plans.

Good stadium nachos

Name me a worse liquor, I’ll buy it next week if I can find it local, and I’ll report back.

Did you go to uni in hell or chicago?

Malort

Tastes like turpentine and grapefruit juice. The former I’ve actually tried accidentally… dipped my paint brush in my cup of water and took a swig of the other cup. Somehow, the malort was worse. Learned recently that they make a barrel aged version that they claim is

dare we say, sippable

We do not.

ssh keypairs secure from the point of authority. In the case of you logging into your account, the server’s job is to ask a question that only you can answer.

In the case of a password, the remote server doesn’t store your password either, it stores a derivative hash that can, in theory, only be generated using complex math + your password as an input. You are the only one that stores your actual password.

In the case of an ssh keypairs, it works very similarly. The public key allows the server to cryptographically ask a question that only the private key can generate an answer to correctly.

It is fundamentally the same secret exchange handshake as a password just with a few extra steps to make things more secure during automation. A password can be weak and guessable, an ssh key cannot (at least not any more than other keys of the same algorithm can).

For music enthusiasts plexamp is also basically unbeatable. I welcome the day open source catches up.

Microvms or containers could give you external control of the networking. Then you would put whatever you want behind warp inside the warp container/vm.

+1 to this observation. I run zfs arrays at both home and work and it’s way more likely that your controller is flaking than you have that many simultaneous drive failures.

The unfortunate reality though is that you can’t trust the current copy of this data, even the snapshots, unless the restore passes a scrub post-restore.

Less cynically, I believe the argument in scripture is the inverse. Man was created in god’s image therefore we probably inherited a lot of properties of the devine.