November 2015 saw the release of a number of highly anticipated games, including Star Wars Battlefront, Call of Duty: Black Ops III, and StarCraft II: Legacy of the Void. (Sadly, the release I was looking forward to the most – XCOM 2 – was pushed back until February of 2016.) Last, but certainly not least, was the release of Fallout 4 – an RPG by Bethesda Softworks that takes place in a post-apocalyptic nuclear wasteland.

To say that Fallout 4 is popular is a severe understatement. A post on GameSpot revealed that the game sold 1.8 million digital copies during the first three days of its release, translating into $100 million in sales. Note that the dollar figure doesn’t include the sale of physical copies!

Given how popular the Fallout franchise is, we thought it would be fun to kick off our case study series by comparing Fallout 4 to its predecessor – Fallout 3. More specifically, we wanted to see what topics were being discussed, and perform a head-to-head comparison of the strengths of each title. Ultimately, we wanted to see if developers had evolved the game to the next level, or if they introduced any problems into the game.

Platform Details

For this case study, we looked at game reviews from all platforms. More specifically, we looked at reviews that came from gamers on PC, XBox and PS4. In terms of games, we looked at Fallout 4, and Fallout 3 (not the game of the year edition, since it contained extra DLC that wasn’t representative of the game upon its initial release).

Automated Discovery

The first analysis we performed was an automated discovery of features. In this mode, Sentiment Radar uses several different techniques to automatically determine what people are talking about. It also ranks the overall sentiment for each feature as well. Here is what it discovered about Fallout 4:

Some observations about each of these categories:

• Game Download – reviewers were talking about the size of the download required to play the game. Some noted that even with a physical copy of the game, they still had to download a large amount of content in order to actually play it. For some with fast internet speeds, this isn’t a problem, but with slow connections and bandwidth caps, this proved to be a bit of an obstacle.

• Release Day – most people were happy on release day, with reviewers noting how highly they had anticipated the game.

• Combat – there were some observations that the combat system had changed, and that it felt more like a first-person shooter than an RPG.

• Perk System – some reviewers felt that the perk system was more superficial when compared to other skill systems.

• PC vs Console – this was split between people who felt the console was better, or that the PC version was better. Some felt that the console versions of the game had to make graphical compromises in order to work well on more limited hardware platforms. Let’s take a look at what Sentiment Radar discovered Fallout 3:

In more detail:

• World Building – gamers felt that the lore and the general makeup of the world was well done.

• DRM – there were some frustrated reviewers who were having problems installing and using the game due to the digital rights management software that shipped with the physical copies of the game. SecuROM in particular was called out.

• Patch – similar to the Bugs/Issues feature discussed in Fallout 4, some felt that the game needed some series of patches in order to make it more playable.

• DLC – most were satisfied with the amount of DLC available for the game, although some advised buying the Game of the Year edition because it came with the extra DLC pre-packaged.

Adhoc Analysis

The automated discovery mode of Sentiment Radar is interesting, but sometimes you need to find out more information about a specific feature. This is where the adhoc analysis comes into play. You can specify what you want to look for as a feature, and have Sentiment Radar perform the sentiment analysis for you automatically. We took some common review criteria (graphics, sound, controls, gameplay, bugs, and price) and some Fallout specific criteria (VATS, quests, combat, power armor) and performed an analysis. Here is what we discovered for Fallout 4:

There were a few unsurprising results:

• Graphics – as expected, people were generally happy with the graphics of the game. This is to be expected, since developers typically take advantage of more recent hardware to deliver more detailed textures and models to the gamer.

• Gameplay – long time fans of the series were not disappointed, and the game played very well with most gamers.

• Quests – at their heart, RPGs are all about the quests that players can discover and complete. Historically, the Fallout series has tried to make the game both free-formed and structured (i.e. you can deviate from the main plot line to pursue other interesting quests), and obviously gamers are happy with how this is being presented.

At the same time, there were some more interesting results:

• Bugs – gamers were generally satisfied with issues and problems with the game. Some noted that they hadn’t yet run into any during their gameplay.

• Power Armor – the power armor is a big topic in the Fallout series, since it makes the player quite powerful. Some gamers were upset due to how scarce the power armor has become.

• VATS – for those who don’t know, the VATS system is a form of target assist available to the player. The game slows down so that the player can target a specific area of the enemy during combat. While there were relatively few comments dealing with the VATS, some of the negative comments touched on how unpowered power armor turns off VATS, and that the game is starting to feel much more like a first-person shooter.

• Price – I was expecting more people to discuss the high price of the game, however, relatively few gamers made mention of it.

Here is what we discovered for Fallout 3:

Most of the same observations from Fallout 4 apply to Fallout 3.

Head-to-Head

We wanted to see how the various review criteria stacked up against each other. Sentiment Radar can perform a SWOT (Strengths, Weaknesses, Opportunities, Threats) analysis between two different products (assuming that they contain the same features). We thought it would be interesting to take the adhoc analysis results from both versions of the game, and see if there were any major differences. Here is the result of the strengths from both games when compared to each other:

In the comparison, we can see that on the features of Graphics, Bugs, Gameplay, Price and Quests, both versions of the game rate similarly with gamers. The big difference comes with Combat and Controls. Generally speaking, from the head-to-head, people prefer Fallout 3 over Fallout 4. As discussed above, the combat engine has changed in Fallout 4, lending itself to being more of a first-person shooter instead of a traditional RPG. Given that the audience of Fallout franchise has been made up of RPG gamers, this rating makes sense.

It might be surprising to see that the Graphics feature is rated nearly identical between the two versions. If you think about it, at the time of Fallout 3’s release, people were generally quite pleased with the graphics engine being state of the art. When Fallout 4 was released, it used a newer state of the art graphics engine. In both cases, the graphics looked spectacular for the day and age where the game was new and was being played by a large audience. We would expect if you compare the two engines today, the Fallout 3 engine would probably start to look more dated.

Conclusion

As you can see, Sentiment Radar was able to discover very interesting pieces of information about Fallout 3 and Fallout 4. When we compare the two head-to-head, we see that Fallout 4 is looked upon less favorably when compared to Fallout 3 when dealing with Combat and Control. Clearly, the shift to a first-person shooter feel has been noticed by gamers who are more traditionally RPG oriented.

Stay tuned for our continuing Case Study series where we will look at other interesting products and see how they have evolved.