Test 1: The Matrix

As usual I reviewed chapters 28-30, that's frames 140155 - 153948 from a total of 196155.

Chapter 28

This is a chapter with very little action. On the other hand we have many close-ups and hard to encode background like uniformly colored walls that have a lot of film grain on them.

Screenshots are always presented in the following order: Original, 3ivX, DivX5, RV9, SBC, WMV9 and XviD. I planned to offer a mechanism to switch between codecs and I will eventually add it once I can manage to properly implement the scripts I was given. Until now the presentation is the same as it always was.

Then my usual disclaimer: Screenshots are not necessarily representative of the video quality. It might be that a screenshot of codec A just happens to fall onto an I-frame which is the least compressed (and thus looks pretty good), whereas quality quickly degrades after the frame shown in the screenshot. And a codec B might have a heavily compressed B-frame at the same position, so comparing the two would be inherently unfair. Bottom line: Take screenshots with a grain of salt, make sure you read all my comments and perform your own tests when in doubt. Every human being perceives visual quality differently and you might come to completely different conclusions watching the same clips as I did.

The major challenge of the first scene is a natural looking reproduction of the walls. There's a lot of mosquito noise in the original which gives each codec a hard time. The buildings which can be seen behind agent Smith are also a challenge for each codec. And during close-ups codecs have to show how natural they can reproduce the human face.

Another keypoint I used were the lights in the ceiling, codecs often seem to consider them not important, resulting in edge problems around them.

Please note that I only mention things when I think they're noteworthy. If a codec doesn't have very visible problems with one of the objects I was looking at closely, I will not mention it.

3ivX: As you can see in the screenshot, there are vertical lines of blocks visible on the wall. The lamps on the ceiling have blurry edges and human faces aren't very detailed.

DivX5: There's a banding effect in a restricted area of the wall.

The lights in the ceiling have somewhat blurry edges and there are a few disturbing blocks in the building with the blue windows.

Close-ups look rather good with few visible blocks.



RV9: Hardly visible on this screenshot, there is a slight banding effect on the walls but the bands were relatively still.

The close-ups look okay, but you can notice a certain amount of detail loss and a block here and then.



DivX3: There's a considerable amount of rather static blocks visible on the walls. During the first shot of Morpheus' perspective you can notice how the building just to the right of Morpheus' arm seems to be periodically "readjusted", an effect which I found quite disturbing.

The building to the right of Agent smith having blue windows seems to be having windows of variable size. According to DivX3, blue blocks of changing size lead to an impression of a size change of certain windows.

Close-ups show a great amount of details, but also a number of blocks.



Additionally there was something which I can only describe as smearing. It looks like sometimes a part of the texture of an object somehow gets stuck to the underlying object and when it moves it drags the texture along. Unfortunately I don't have a shot of this to show you but it was repeated throughout the test.

WMV9: There's a visible banding effect on the walls. The building to the left of Morpheus' shoulder (the same SBC had problems with) seems to flicker.

The building with the blue windows is very smoothed out, it's hard to make out any details whatsoever.

During close-ups you can sometime notice an edge problem.. it appears as if a block is "sticking out" of the face. And while a certain amount of smoothing can be noticed there's still an occasional block to be seen.

XviD: There's a very slight banding effect on the walls, and the bands move around a bit.



It is interesting to see that this scene shows different effects than in earlier comparisons. Instead of having static or dancing blocks on the walls, modern codecs mostly seem to exhibit a banding effect, which isn't quite as disturbing as blocks, but the less of it the better. All codecs performed reasonably well during this scene.

This shot is an indicator of the general detail level. A close-up with very little colors used, you can immediately spot which codec smoothes out details and which ones try to conserve them at the cost of eventual blocks.



3ivX smoothes out some details while retaining a rather harmonic impression of face details.

DivX5: As you can see, DivX5 still smoothes out details. This effect is especially visible (even when you watch the movie at full speed) in the nose and eyes area. Additionally, face edges are not always very clear.

RV9: As you can see, the face is rather detailed. This is an effect that can be seen throughout the test: the foreground is usually rather detailed, whereas there aren't many details in the background. As in this shot the background contains virtually no details, the codec can shine.

DivX3: There's a reasonable amount of details on the face, but also an occasional block or two.

WMV9: Also retains a reasonable amount of detail. Though this screenshot does not do the codec justice.. on my review sheet I noted "Neo's face: almost no details".

XviD: Keeps a good amount of details while not being blocky.

Maybe this is a good opportunity to mention something I noticed with XviD throughout the test: QPel appears to help with details, but it also introduces an effect that I don't think is really wanted: Especially in areas with great details (for instance Neo's right cheek), the details seem to be having a life of their own. They seem to move around just a tiny bit. During most scenes this effect can hardly be seen but when there's very few objects that are shown close up, like this example, it's visible. I recall the same effect from DivX5 when I was testing out QPel there. I decided that QPel did more harm than good in early DivX5 versions. In the tested XviD build, these effects which I like to call QPel artifacts, were not so visible. I'll eventually encode the movie again without QPel to verify if my hunch regarding this effect is correct or not.



When it comes to close-ups, RV9, SBC and XviD keep the largest amount of details, with SBC exhibiting some blocks.

This is what I like to call a killer scene. The fast zoom into the weapon's rack, and the rapid movement of the rack behind Neo make it hard for any codec to make a good impression. Not even the original looks too great.

All codes showed the following effect during this scene: There was a periodic readjustment of the image quality as a new I-frame is used. This can be seen by a periodic improvement and then degradation of the image quality.



3ivX: exhibited the aforementioned effect.

DivX5: Also exhibits the abovementioned effect. The shot shows a frame that's in the bad quality area and is not representative of the overall impression.

RV9: made a pretty good impression in this scene. The shot shows why: Neo isn't shown very detailed so bits could be used otherwise.

SBC: Same effect as with all codecs.

WMV9: also exhibited the abovementioned effect. As can be seen, detail was traded for a more harmonic global impression.

XviD: In my last test I heavily criticized XviD for this scene. Apparently it has helped because while the quality adjustment effect is still very visible, the codec no longer comes in last and performs better than DivX5 and SBC.

The rest of this chapter was pretty uneventful so I decided not to include another close-up shot. There's just one noteworthy thing: After the weapons rack comes to a stop, the quality goes up again in all codecs but 3ivX. It seems that 3ivX has exhausted its allocated bit reservoir and during the rest of the chapter (and in some way the rest of the following chapters I reviewed) a significant quality degradation could be noted. Since I also noted this effect in SPR (including a quality readjustment after a certain while) I suppose a 2 pass encoding mechanism would help 3ivX to provide more constant quality.

There's not much to be said for the next scene. Matrix wouldn't be Matrix without the lobby scene. I'm just wondering if I have to switch to an even better scene when the next movie comes out. Having watched the trailer over and over again I'm expecting some great mayhem scenes.

Even in the last test, codecs generally did really well during the lobby scene and made it hard to find differences, so I decided to reduce the number of screenshots for this comparison as they really don't say much.

The scene shown on the left is ones that can still be used to make a difference.

The beginning of the scene where many policemen come running in the lobby is also a good indicator (have a look at the pillars).



Just when you'd think 3ivX has recovered from the last bitrate overuse shock comes the lobby scene. Here's a quote from my evaluation sheet: "Am I watching a VCD?". The answer is of course no, but 3ivX bottomed out during the complete chapter.

I didn't have much to write about DivX5. It makes a rather good impression, with slightly less details than both SBC and XviD.

RV9 also makes a good impression, but this shot proves that RV9 saves bit by blurrying certain elements.

SBC performs well but has the largest amounts of blocks. And the pillars before all the shooting breaks out don't look too good. And even during the action, sometimes large parts of the screen become blocks for just a second (good thing I didn't use ffdshow because it didn't do such a good job at restricting this effect and would've resulted in DivX3 getting a lower score in many scenes).

WMV9 didn't make such a shabby impression either. I found it to be less detailed than the DivX3/5/XviD combo, and I noticed a loss of detail when there is smoke on the screen.

Finally, XviD also made a good impression, but when there's a lot of debris flying I noted the QPel effect.

Then we have a close-up scene:

Even the original doesn't look too great which shows how difficult it is to adjust the bitrate to a slow motion scene that still requires a high bitrate due to details on a human face.



No comment about 3ivX, the shot speaks for itself.

DivX5

RV9

Even on the screenshot you can see that DivX3 preserves more details than most other codecs.

WMV9

XviD

And on a final note to this chapter: If you see people up close you'll usually notice a lot of blocks in the background. This effect is most visible in SBC. Have a look at Neo's coat when you see him from behind, or when you see the close-up of one soldier shooting with at Neo with an M-16 rifle. And the elevator door looks blocky in SBC as compared to the other codecs. RV9's focus on foreground objects and keeping a rather static background pays off in such scenes.. you may notice the lack of detail when looking hard enough but it's not surprising that some people like RV9 a lot.. if you don't pay attention to details like the background RV9 makes the most harmonic impression but it's closely followed by XviD and WMV9.

The last chapter has everything. Still scenes, scenes that will go from slow to action packed within seconds, a huge explosion, water, everything it takes to give a codec a good beating. First we start off with a still scene that once again exhibits what I mentioned in the first chapter. The elevator walls and the level indicator in the elevator are also interesting points. SBC performs worst in those areas, exhibiting visible blocks.

Boom!



This shot doesn't do 3ivX justice, the general impression was a lot worse.

DivX5

RV9

SBC. In the followup up scene, the moving texture syndrome can be noted in the lower left of the screen as the fire approaches the camera.

WMV9

XviD

The next one shows how fast the bitrate can be increased to cope with a difficult situation.

The water, and especially the lights going out before the sprinklers are turned on, create a good codec killer scene.



3ivX doesn't need a comment...

DivX5 has some problems with the lines on the wall when the lights go out and there are some blocks once the sprinklers are turned on.

You can see how much RV9 eliminates details in this scene.

In DivX3 we have many blocks on the walls in general, and especially when the lights go out.

There's some banding visible on the walls in WMV9, similar to DivX5 but the scene looks slightly better.

There are a few blocks visible as the lights go out The water from the sprinklers also exhibits an occasional block or two.

Let's now conclude this chapter.

The lightning is also a killer for every codec I tested so far.

Before this, Neo falling to the floor also exhibits some blocks in certain codecs (WMV9, RV9).



3ivX

DivX5 has some visible blocks.

The building behind the agent when he's firing at Neo is now shown in great detail compared to SBC and XviD.



RV9

SBC

WMV9 also gets blocky despite the screenshot showing otherwise.

Also note that WMV9 doesn't perform too well on the roof. The red ACM sign on one of the buildings (when Neo's on the floor and the agent is coming at him) cannot be deciphered.

The building behind the agent when he's firing at Neo is also not very detailed, the distinction between the various windows cannot always be made.

XviD didn't have many blocks anymore, which is quite an improvement compared to earlier versions of the codec.



In conclusion, we have been able to see that at 600kbit/s no codec can even get close to DVD quality.

Once again, differences between codecs have been decreasing and were it not for 3ivX all codecs would've played in the same league. RV9 and WMV9 have the most visible tendency to smooth out details. On the other end of the spectrum, SBC and XviD tend to show the largest amount of details. DivX5 is situation in the middle of those two factions.

Personally I value details a lot, and were it not for the QPel effect in XviD, which I found to be slightly disturbing in some scenes, I'd be inclined to say XviD handles Matrix best, hands down. As it is, XviD is up to par with SBC on details while being less blocky which makes it a very valid alternative.

Now let's move on to Saving Private Ryan. If you think you've seen enough you can proceed directly to the conclusion.

This document was last updated on May 6, 2003