AWS re:Invent 2024, Oracle Cloud AI, GenCast, videos

AWS re:Invent 2024 – Monday Night:

  • Graviton evolution: ARM based chip for EC2. 50% new capacity of last 2y is Graviton.
  • Nitron Cards: security chip too.
  • AES Trainium2: min 47. 2xHead per rack and then accelerators, and switch. Trainnium != CPU|GPU
  • Neurnlink: min 60, I guess this is the equivalent of NVLink, etc
  • Ultraserver, quite beefy pic, min 61.
  • Networking: min 73: 10p10u is a fabric = 10petabits under 10micro latency.
  • Cabling proprietary trunk connector 16:1 fiber. min 77. I pretty use i have used pig-tails some years ago, so not sure why this is new?
  • Firefly optic plug: loopback testing. This is interesting for DC operations. Min 78.
  • AWS design their own optics, reduced failure
  • Network topology: Min 81, new protocol SIDR – Scalable Intent Driven Routing. <1s reconvergence. not centralized.

AWS re:Invent 2024 – NET201: The only interesting thing is minute 29 with the usage of hollow core fiber, to improve latency. I assume it is used in very specific parts of the network, looks a bit fragile. Elastic Fabric Adapter, not really good explanation what it is, where doest it run: network, server, nic? but it seems important. Looks like SIDR?

Oracle Cloud Infra – AI: First time I visit the OCI page about their AI infra.

GenCast: weather predict by Google Mind. Not sure until what point, this can be used by anybody? And how much hardware you need to run it?

we’ve made GenCast an open model and released its code and weights, as we did for our deterministic medium-range global weather forecasting model.

Videos:

510km nonstop – Ross Edgley: I have read several of his books and it is the first time I watch a full interview. Still I am not clear what his dark side is.

A man with few friends or not circle at all – Jordan B Peterson: I need to watch this more often

Discipline is Destiny

Very good book, as discipline keeps me in my place and sane. And nobody is perfect, the key is to get back up when failing.

This book is a very good reminder of Jocko Willink: Discipline = Freedom.

Persist and Resist

1) The Exterior (The body)

  • Ruling over the body: Gives the example of Lou Gehrig, I have no clue about baseball but it is very interesting how he kept at the top, no noise, and died so young.
  • Attack the dawn: wake up early (and go early to bed)
  • The strenuous life is the best life: Train your body, take care of your body
  • Quit being a slave: of craving, addiction, vices
  • Avoid the superfluous: – desire -> + rich
  • Clean up your desk
  • Just show up
  • Sweat the small stuff
  • Hustle, Hustle, Hustle: I dont really understand this part.
  • Slow down… to go faster
  • Practice… then practice more.
  • Just work
  • Dress for success: Interesting facts about Angela Merkel.
  • Seek discomfort
  • Manage the load
  • Sleep is an act of character
  • What can you endure?
  • Beyond the body

2) The inner domain (The temperament)

  • Ruling over yourself
  • Look at everything like this
  • Keep the main thing the main thing: Learn to say no.
  • Focus, focus, focus
  • Wait for this sweet fruit
  • Perfectionism is a vice
  • Do the hard things first
  • Can you get back up?
  • The battle against pain
  • The battle against pleasure
  • Fight the provocation
  • Beware this madness
  • Silence is strength: Example from Sparta.
  • Hold, hold your fire
  • Temper your ambition: Example from a young Napoleon noticing the evil of ambition and then later in life he ignored his own advice.
  • Money is a (dangerous) tool
  • Get better every day
  • Share the load
  • Respect time
  • Put up boundaries: Examples of the Queen Elizabeth II
  • Do your best: Example from a young Jimmy Carter, when he recognized not doing his best when he had an interview with a general.
  • Beyond the temperament

3) The Magisterial (The soul)

  • Elevating yourself: Antoninus Pius was the mentor and stepfather of Marcus Aurelius
  • Tolerant with others, strict with yourself
  • Make others better
  • Grace under pressure
  • Carry the load for others
  • Be kind to yourself
  • The power of giving power away
  • Turn the other cheek
  • How to make an exit
  • Endure the unendurable
  • Be best
  • Flexibility is strength
  • Unchanged by success
  • Self-discipline is virtue. Virtue is Self-discipline

TPUv6, Alphafold, OOB design, OpenInterpreter, Walkie-Talkies, Zero Trust SSH, Videos, Finger Strength

Google TPUv6 Analysis: “… cloud infrastructure and which also is being tuned up by Google and Nvidia to run Google’s preferred JAX framework (written in Python) and its XLA cross-platform compiler, which speaks both TPU and GPU fluently.” So I guess this is a cross-compiler for CUDA?

“The A3 Ultra instances will be coming out “later this year,” and they will include Google’s own “Titanium” offload engine paired with Nvidia ConnectX-7 SmartNICs, which will have 3.2 Tb/sec of bandwidth interconnecting GPUs in the cluster using Google’s switching tweaks to RoCE Ethernet.” So again custom ethernet tweaks for RoCE, I hope it makes to the UEC? Not sure I understand having a Titanium offload and a connectx-7, are they not the same?

Alphafold: It is open to be used. Haven’t read properly the license.

OOB Design:

Open Interpreter: The next step in LLMs is to control/interact with your system.

In my laptop fails because I have the free version 🙁 need to try a different one, but looks promising!

open-interpreter main$ interpreter --model gpt-3.5-turbo



Welcome to Open Interpreter.

───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

▌ OpenAI API key not found

To use gpt-4o (recommended) please provide an OpenAI API key.

To use another language model, run interpreter --local or consult the documentation at docs.openinterpreter.com.

───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

OpenAI API key: ********************************************************************************************************************************************************************


Tip: To save this key for later, run one of the following and then restart your terminal.
MacOS: echo 'export OPENAI_API_KEY=your_api_key' >> ~/.zshrc
Linux: echo 'export OPENAI_API_KEY=your_api_key' >> ~/.bashrc
Windows: setx OPENAI_API_KEY your_api_key

───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

▌ Model set to gpt-3.5-turbo

Open Interpreter will require approval before running code.

Use interpreter -y to bypass this.

Press CTRL-C to exit.

> what is my os?
Traceback (most recent call last):

Walkie-Talkies: Out of James Bond world.

Zero Trust SSH. From Cloudflare. And this video I watched some months ago (and it is already 4y).

Finger Strength: I follow similar protocol, although not everyday, for warm up and I think it works. I am not getting that super results but at least my fingers are stronger…. and I am not getting injuries!!!! \o/

Cisco AI/ML DC Infra Challenges: I am not quiet fan of Cisco products but this is a good overview.

Key points:

  • Create different networks (inter-GPU, front-end, storage, mgmt),
  • Inter-GPU:
    • – non-blocking, rails-optimized (fig.3)
  • Inter-GPU challenges:
  • – Packet loss: Use PFC +ECN (flow aware)
  • – Network delay: “Rich” QoS – proprietary QoS to handle mice flows. Needs good telemetry
  • – Network congestion: Some kind of communication switch-NIC
  • – Non-uniform utilization: Most vendors have something proprietary here, some dynamic LB and static-pinning?
  • – Simultaneous Elephant flows with large bursts: dynamic buffer protection (proprietary)

Videos:

  • Raoul Pal: Crypto Investment. His company. Go long run, invest a bit you can lose
  • Scott Galloway: Interesting his political analysis. Trump won and it seems Latins voted massively for him.
  • Bruce Dickinson: I read Bruce’s books some years ago so I was surprised to see him in a podcast. Need to finish it.
  • Eric Schmidt: I read one of his books some time ago so again, surprised to find him in a podcast. Still think Google has become evil and most of the good things he says are gone.
  • Javier Milei: I am not economist but it “seems” things are improving in Argentina. He is a character nonetheless. Need to finish it.
  • Matthew McConaughey: His book was really refreshing, and seeing him talking is the same. Raw, real.
  • Alex Honnold: You have to try hard if you want to do hard things.

Membrillo

Last Saturday while walking in the neighbourhood, I saw quince and decided to buy some to make membrillo, as it has been a long time since last time I did it. And I noticed I didn’t have a recipe for that online. I used to follow the recipe from this book, but honestly, it was never like the one my mum used to make when I was a kid.

So I decided to find a video and try again as I don’t have the book with me. A decided for this one as I liked her video about Tortas 🙂

Ingredients

  • 4 quince/membrillo – 850g aprox after boiling
  • 500g sugar
  • Juice from one lemon
  • 1 or 2 glass jars. Cleaned with boiling water

Process

  • Peel the quince, remove the core as it bitter. You will think that you are removing a lot but that is how it is. Chop in pieces
  • Put the quince in a big pan/pot. I don’t have express cooker. Cover in water and boil until it is soft. Use a knife, you should cut through the quince with ease.
  • Drain the quince and weight it. Mine was 850g so I added 500g white granular sugar. Some other recipes add 1:1 quice/sugar but I didnt want to add that much.
  • Add the juice of one lemon
  • With a hand mixer, make a pure .
  • Then starts the longest part. At low heat (use a glove!!!) start stirring the mix until it gets a dark orange color. Be sure it doesn’t stick! Keep in mind that it has a lot of sugar… and it is like molten lava when cooking! At the beginning the mix is white/yellow-ish, I think after 1h, it started to change the color and it is when you have to be very alert and stir often (each minute) to avoid burning.
  • When you use the wooden spoon to make a line and it doesn’t come together, it should be ready
  • Remove from heat and poor it in glass containers if possible, if not plastic.
  • Let it cool down during the night at least.

And this is the result from this morning. This is like it was when a kid!!!!

Borges

In my last day in Argentine last year, I bought a small book from Borges as I wanted some memory of my trip. I think it is poetry but I dont understand it. I have never been able to read poetry, even in my mother tongue.

The book contains “Fervor de Buenos Aires”, “Luna de enfrente” and “Cuaderno San Martin”.

Maybe once day i will be able to capture the beauty of this type of literature. As a teenager, I remember one teacher explaining us some poems from Pedro Salina’s “La voz a ti debia” And it was really nice, I still go the book, pending to read it…. nearly 30y in the waiting 🙂

Hyperfocus

I guess I am bit obsessed with personal performance, how to make more and better in the same or less amount of time. And this is another book about the subject. I read it as ebook so didn’t take notes.

Although the main topic is how to focus and make the most of it, the second part is about “scatterfocus” that was unexpected.

Focus produces, scatter invents/solves. They follow different steps to achieve that state.

And I think focus is gold. Difficult to focus with so much distraction everywhere. I like to work in the office… because it is mainly empty and I have few distractions there but looks like everything is about attention. It is the real commodity. Everybody is fighting for it. So you have to look after it.

There is nothing revolutionary for getting focus, and that is a good thing. I like the emphasis in meditation. Be sure you look after your body: good sleep, exercise, proper food, and be sure you have a limited time to be real focus per day so make it count.

For scatterfocus, it is as easy as going for a walk without nothing really in mind, let the mind wander and think in things and problems. These are the moments when most eureka bursts happen.

Need to read it again.

Dead’s End

I completed last week the third part of the Three Body Problem trilogy. It was interesting but in this part things got wild! Every twist caught me out of guard. The physics got me lost most of the time (3D vs 4D, etc). But I like how Trisolarians learnt to lie and play long term. How reaching light speed was the proper solution and the most important thing: humans in a spaceship disconnected from Earth, took them 5 minutes to reach Totalitarianism. And all the theories about the Dark Forest hypothesis (Fermi Paradox), deterrence, and something a bit less sci-fi.

Destroying Earth using a 2D plane that sucks the surrounding 3D… but that would suck the whole universe… but that is a good tactic if you can live in 2D… not sure if that was a superior tactic to defeat your adversaries in the dark forest?

SemiAnalysis – 100k cluster

This is site that a friend shared with me some months ago. And it is PURE gold from my point of view. They share a lot info free but not all, you have to subscribe/pay for the whole report. I would pay for it if my job were in that “business”

This is the link for a 100k GPU cluster.

It covers all details for building such infrastructure up to the network/hardware side. So from power distribution, cooling, racking, network design, etc. All is there.

It is something to read slowly to try to digest all the info.

This report for electrical systems (p1) shows the power facilities can be as big as the datacenter itself! So it is not rear to read hyperscalers want nuclear reactors.

MS GB200 rack, Malaysia DC boom, Oracle DCs, FuriosaAI, OCP concrete, IBM Mainframe Telum II, NotebookLM youtube summary, EdgeShark, RSA Quantum, OCP24 Meta

It seems Malaysia is getting a DC boom, but it based on coal???

This is a MS NVIDIA GB200 based rack. I am quite impressed with the cooling systems being twice as big as the compute rack! And yes, MS is sticking with IB for AI networking.

I didnt know that Oracle OCI was getting that big in the DC/AI business. And they were related to xAI. Their biggest DC is 800 megwatts… and a new one will have three nuclear reactors??

FuriosaAI: A new AI accelerator in the market. Good: cheap, less power. Bad: memory size.

OCP concrete: Interesting how far can go the OCP consortium.

IBM Mainframe Telum II: You think the mainframes business doesnt exist. Well, it is not. Honestly, at some point, I would like to fully understand the differences between a “standard” CPU and a mainframe CPU.

NotoebookLM: It seems it is possible to make summary of youtube videos! (and free)

EdgeShark: wireshark for containers. This has to be good for troubleshooting

22-bit RSA broken with Quantum computer: I think Quantum computing is the underdog in the current “all-is-AI” world. Schneier says we are ok.

OCP24 Meta AI: It is interesting comparing the Catalina rack with the one from MS above. The MS has the power rack next to it but FB doesnt show it, just mention Orv4 supports 140kW and it is liquid cooled. I assume that will be next to Catalina like MS design. And AMD GPU are getting into the mix with NVIDIA. It mentions Disaggregated Scheduled Fabric (DSF), with more details here. And here from STH more pictures.

A Man for All Markets

Very interesting book. I learned a lot new things from the origin of card counting, beating the roulette with the first wearable device (with Claude Shannon!!!), beating the markets based on managing risk, etc. The author is truly remarkable.

It is interesting how Edward moved from Chemistry to Maths due to problems with fairness in this Chemistry “career”. And still he didnt find much fairness in Las Vegas, and even worse in the stock market. He was driven to solve problems people didnt think had a solution. And was impressive how he taught himself. And likely he was a pioneer in computer-based trading. He is against the low latency trading. He mentions many times Warren Buffet and his investment style. As well, Citadel, as his continuation about the management of risk. It seems he didnt look for the sort term profit but going long, looking for mis-priced stocks.

I am happy he doesnt believe in the efficient market neither.