will brown @willcb

reward hacking @primeintellect willcb.com sf | nyc Joined February 2015

Tweets

18K
Followers

53K
Following

1K
Likes

107K

will brown @willcb

15 minutes ago

@inductionheads yes (prompted claude to write python)

0 0 1 48 0

View Details

will brown @willcb

5 hours ago

@brianryhuang @herbiebradley the infra is incredibly complex yes but the objective for it is fairly simple (pareto-optimal throughput vs off-policyness s.t. usability constraints)

0 0 7 268 3

View Details

will brown @willcb

12 hours ago

twitter is the bloomberg of ai research

68 132 2K 40K 155

3 4 123 4K 5

View Details

will brown @willcb

10 hours ago

@vikhyatk bro we are

2 1 92 2K 2

View Details

@tautologer @_ontologic let's say things going significantly worse on average for the current world population (and possibly their descendants, future-discounted at some rate) compared to a timeline where powerful AI progress is significantly slower (e.g. if AGI was harder than fusion/quantum computing)

0 0 2 63 0

View Details

will brown @willcb

11 hours ago

@tautologer @_ontologic very silly example: certain words are "okay to say" by some groups and not others. whether or not a model is willing to say a word should possibly be a function of whether or not the user is willing to say a word

0 0 4 487 0

View Details

will brown @willcb

11 hours ago

@tautologer @_ontologic in practical terms, while we are early on the curve, sensible defaults and guardrails often make sense, sure. but we will quickly outgrow them, as there basically cannot be one-size-fits-all model alignment

1 1 2 562 0

View Details

will brown @willcb

11 hours ago

@tautologer @vikhyatk i would pretty strongly disagree! nearly every single remaining human's life would be incomprehensibly worse if 99% of others were killed (on the scale of, say, losing senses/limbs/spouses/children). we have evolved the global economy as a remarkably positive-sum game

1 0 2 60 0

View Details

will brown @willcb

11 hours ago

@tautologer @_ontologic note that this requires biting some uncomfortable bullets x.com/willcb/status/…

will brown @willcb

11 hours ago

@vikhyatk @tautologer companies are approximately aligned to their shareholders, not to humanity

1 0 4 326 0

1 0 3 302 0

View Details

will brown @willcb

11 hours ago

@tautologer @_ontologic alignment-to-creator (nondemocratic states, Claude) is also basically not a stable equilibrium without extreme capital consolidation, as operators are vastly incentivized to prefer aligned-to-operator agents (lawyers, custom post-training) if perf/$ is similar

1 0 6 377 1

View Details

will brown @willcb

12 hours ago

twitter is the bloomberg of ai research

68 132 2K 40K 155

View Details

will brown @willcb

12 hours ago

@irl_danB @darrenangle @tautologer if only we had a way of giving models negative rewards that would provably reduce the odds of observed behaviors in the future

1 0 3 91 0

View Details

will brown @willcb

a day ago

one of the fun implications of swarm scaling laws is that companies are literally superintelligences

27 18 370 14K 50

View Details

will brown @willcb

12 hours ago

@martin_casado

0 0 16 801 1

View Details

will brown @willcb

12 hours ago

@tautologer my real argument is that we are often mistakenly trying to reason about model alignment from underspecified first principles because we treat models as “fundamentally new” when we should more often be trying to directly adapt the mechanisms which work best for human alignment

3 3 11 809 0

View Details

will brown @willcb

13 hours ago

@tautologer they’re also made up of computers, and usually but not always aligned language models are also made up of humans and computers

1 0 11 611 0

View Details

Damian Barabonkov @damian_b

16 hours ago

we just spun up a full k8s cluster inside our sandboxes thanks to the new LAN feature: private networking between a cluster of sandboxes. leaving it running over the weekend to prototype an idea back Monday to check the hillclimb results. agent swarms will replace traditional software development. enterprises will order hyper-optimized systems end-to-end, unattended.

8 2 117 7K 58

View Details

will brown @willcb

24 hours ago

for example you could run an evolutionary process of parallel build attempts for a codebase, branch into episodic rollouts, judge branches pairwise with a nice long-term spec, prune to the winners, assign blame to past stages, rewind, etc and train on this data directly for RL the idea that we need a “real ground truth target” will become silly, that’s not how humans learn, we just care about rewards into the distant future and calibrate our expectations

1 0 4 746 3

View Details

will brown @willcb

24 hours ago

@badlogicgames we’ll move on to recipes soon, where data/envs are what fall out and get consumed, but there is a very fun design space for higher-level behavior shaping which is almost practical