Originally, rumors had us waiting three weeks until Sony's E3 presentation before update 2.4 with in-game XMB had come out. Sony has officially confirmed that the update will be released on July 2nd. In-game XMB is pretty self explanitory. When playing a game, you'll be able to access XMB functions rather than having to quit the game first. This is perhaps the most demanded non-game ability for the PS3 after the DualShock 3. Gamers mainly wanted it so they could access the chat and messaging functions for PSN while they were playing games. If you're playing with friends online, this obviously makes sense. Microsoft's Xbox Live has had simmilar in-game messaging for some time now. Some other features with in-game XMB allows for custom soundtracks. It's pretty primitive right now. Basically, you can play your own music and playlists over a game complete with a mini-controller. Settings can also be changed now in game. Say I'm playing a game and I want to change the sound from HDMI over my TV to RCA on my stereo, now I can do that without having to quit first.

Two other included XMB features are date and time, which will appear in the top right corner of the screen like on the PSP. Also thrown in for kicks is a rather useless Google search option under network as was done in PSP update 4.0. (Useless since you can just go to Google using the web browser)
Also added is a trophy system similar to the one used on Xbox Live. This will allow you to record acomplishments in some games. Currently, not a lot of titles are supported. Slated for trophies so far are LittleBigPlanet, SuperStarDust HD, BUZZ! Quiz TV, MotorStorm Pacific Rift, NBA 2009, PAIN, PixelJunk Eden, Resistance 2, SOCOM: Confrontation, and Warhawk. Trophies can be shared with friends over PSN. There are a couple of official blog posts complete with videos that explain how all this works.

Walkthrough Part 1: The XMB
Walkthrough Part 2: Trophies

The second post confirms the date. Yes, I promise I do have some non-PS3 stuff in the works.

Update: July 2nd
I had a chance to play around with 2.4 this morning. I can't exactly say I'm impressed. For starters, Sony was kind of vague on the abilities of the in-game XMB so I figured I'd be able to do certain things based on what was said in the videos. For example, as I mentioned above, I thought you could adjust any system settings in game. However, in reality, you can only adjust settings for "accessories" such as Bluetooth devices. All other settings such as sound and display require you to quit the game. Also, Sony indicated that I would be able to play my own music in game, any game from the way it sounded in the video. I loaded up Gran Turismo 5 Prologue thinking I could play my own driving music. A message popped up saying that "music content isn't available for this game". I tired Uncharted and I couldn't play my own music for that game either. I figure it's the same for all my games. So apparently, games must support that feature and it's not as easy as simply turning down the in game BGM volume as the video showed.
Furthermore, in-game XMB does not work for PS1 and PS2 titles, which was only revealed in an interview with Gamespy but not mentioned in any of the official material. The new clock is a handy feature but it also doesn't appear in PS1 and PS2 games when you press the PS button. I'm getting the feeling the in-game XMB feature may have been rushed. It has a very incomplete feel to it. While it's interesting overall, it's pretty useless except for chat. Lastly, the XMB from within games has some lag in switching menus, equal to what you'd get using remote play over your PSP. Overall, this update is pretty disappointing. As the old lady said, "where's the beef?" Since I don't own any of the games that currently support trophies, I can't comment on that feature.

Update 2: It appears Sony has "temporarily" pulled firmware 2.4 from the internet due to a "limited number" of issues. Apparently it's bricking some systems. I personally haven't experienced any issues but people in the PS community are recommending you back up your saved data in case this happens. Why it would effect data stored on the HDD is a mystery to me but it's always a good idea to back up any computer system before major updates. Apparently if you do have a problem, it will happen right away, so if 2.4 installed successfully, you're alright. Over on PSU, there's rumours that Sony will release FW 2.41 in 24-hours to patch the issues.
The Inquirer is claiming Sony's Playstation website has been hacked and some in the PS community are saying this has something to do with the 2.4 issues. I dobut it, and besides, anything from The Inquirer should be taken with a grain of salt.