Review Schema

[Freakish]

In order to try and keep reviewing as fair and objective as possible, we have developed a set of categories and rules that every reviewer follows in exploring the successes and failures of a map. The goal of this system is to remove as best we can the reviewer bias that so often creeps into other reviews.


--What's this about team reviews?

UnrealPlayground has added an innovative twist to the reviewing process. Every Playground reviewer is qualified to review for Gameplay, Design, or both. This is to accommodate staff that may be best suited to only one part of the review (example: a player who doesn't know UnrealEd or Design very well can still help with reviews since he/she is an excellent critic of Gameplay). Therefore, reviews can be done by one person (who can competently do both parts) or two. This also helps us work feedback from mappers into reviews; mappers can comment on Design without having to worry about writing a full-fledged review.


--Two Types of Reviews?

There are two types of reviews here at the Playground: Features, and Regular reviews. As suggested by the name, Feature reviews are more in-depth than ordinary reviews. Only maps deemed excellent (scoring over 80, preferably at least 90) on the scale are eligible for being Features. Playground staff choose Feature maps out of the pool of maps waiting to be reviewed.

So why are normal reviews so much shorter and less complicated than Feature reviews? Well, in general we want you to be able to read through a review quickly to get a feel for a map. We feel that the simpler two-category approach (Gameplay and Design) helps condense the information you need to know in making your decision about whether or not a map is worth your download. While long reviews get more information across, they also are usually too much information for the casual reader. Our system helps keep reviews short enough that the casual reader can spend little time going over a particular map, while long enough that the reader can get the main points that would be presented in each category of longer reviews.

As Feature reviews are reserved for the cream of the crop, these maps naturally deserve fuller reviews. Therefore all categories are separated and displayed individually, complete with in-depth contents for each. This helps mappers and casual readers alike see just what makes up a great map.


--Requesting Reviews

Currently we process review requests as we receive them. Naturally, our reviewers are free to bypass the review queue and choose a map they feel deserves a review, but in general reviewers are encouraged to process the list in order and simply add any maps they wish to review to the end of the list.

Please realize that review requests are plentiful and reviewer time is fixed. We'll process the requests as best we can, but be prepared for a wait.

Before you request a review, upload your map to the Playground. OUr database requires the map to be onhand on the site. We don't have time to scour the Internet looking for that gem of a map you want reviewed.

Also be sure that the zip includes all files necessary to run the map. Maps that require additional textures or files not included in the zip are generally declared 'unplayable' and will not get reviewed. If we decide your map is still worth our effort despite the missing file(s), its score in the Technical Playability subcategory will be penalized.

You can learn more about requests in this thread.

And finally, please remember that UnrealPlayground staff always reserves the right to refuse a review without reason. We do not anticipate this being anything more than a rarity, but in certain special cases this may happen.

--Discussing Reviews

Please do! Ask reviewers to elaborate on why they scored the Gameplay so low even though they said it was extremely fun. Players can give their opinions on the map in question, or the review in question. If you think a review was biased, voice your opinion. If you completely agree with a review, post your thoughts so the reviewer and mapmaker know the opinions of the reviewer are shared by others. To discuss a review, simply check for the appropriate thread in this forum. You can be assured that reviewers check the forums and will most likely respond to any constructive comments you have.


--The Nitty Gritty Details

So, if you've read this far, then you probably want to know exactly what the categories and their weights are. The categories are the same for regular and feature reviews; but in regular reviews the sub-categories that make up the two overarching areas of Gameplay and Design are hidden from the viewer. The following should be self-explanatory. Please note that some comments below do not apply to all gametypes.

###Overarching category: GAMEPLAY
Weight: 70%

#Sub-category: Enjoyment
Weight: 35%
Description: Is this map fun the first time you play it? How about 10 rounds later? Just how replayable is it? Are games generally great on this map? Is there something about it that really makes it fun even when teams are off or you're not playing well?

#Sub-category: Flow
Weight: 15%
Description: Does the layout confuse people? How much of a learning curve is there? Is the learning curve steep enough to detract from enjoyment? Is the map intuitively laid out? Can players navigate the map easily? Do obstacles get in the way during the game? Are spawn points reasonable or do players have to travel great distances just to grab a weapon or get into the action?

#Sub-category: Technical Playability
Weight: 20%
Description: How's the FPS? Are there BSP errors; either visual (ie. Hall of Mirrors effects) or physical(BSP holes, those great spots that result in instant-death)?

#Sub-category: Artificial Intelligence
Weight: 20%
Description: We all know the bots are about as smart as cantaloupes. But, how well did the author cope with this? Do bots get stuck in the map? Do they obey orders? Can they even cap the flag or get to a control point? How well do they respond when they are under fire and have the flag? Can they get the items, do they have a variety of paths to take? Can they find a human opponent hiding with the flag?

#Sub-category: Item Placement
Weight: 10%
Description: Is there enough ammo and health? Is weapon choice reasonable? Are there too many powerful weapons or items together? Is there a consistency in placement (ie if some weapons are laid on the ground, are there others floating and spinning)?


###Overarching category: DESIGN
Weight: 30%

#Sub-category: Theme
Weight: 25%
Description: If you hadn't read anything about the map, would you be able to determine its theme from exploring the map? Does the theme make sense? How is the theme conveyed to the player (is it just texturing, or do architecture, sounds and lighting also contribute)? Do you feel immersed or detached from the environment?

#Sub-category: Architecture
Weight: 20%
Description: How complicated is it? Is there a reason behind its complexity? Does it reflect the theme? Is it consistent throughout the map? If not, was the author lazy or clever?

#Sub-category: Texturing
Weight: 20%
Description: Are textures placed smoothly throughout the map? Are there noticeable misalignments or clashes? Do textures make sense when taking theme into consideration? Are they ugly? Do they inhibit gameplay by making it difficult to see players, items, hazards, the general layout, etc?

#Sub-category: Lighting
Weight: 20%
Description: Is the map even lit enough to see? Is the map too bright? Are there fixtures to go with light sources, and if so, do the sources make sense coming from the fixtures? How well is colored lighting used? Do fixtures make sense under the theme?

#Sub-category: Sound Effects
Weight: 15%
Description: Do sound effects add to the atmosphere? Do they distract you from the game? Are they too loud or soft (especially if they are supposed to emanate from specific objects or locations)? Do they serve any purpose or are they just there because the author thought it would be cool to have a touchpad make a farting noise?


So how does this all come together? The subcategories are weighted into each overarching category. These two overarching categories are then weighted and combined into the final review score. Each sub-category and overarching category is scored on a scale of 0-100 (just as the overall score is). An explanation of the scale follows..

For each category, the following applies.
0-20: Horrible. Whatever it is, it doesn't work or has serious errors that affect gameplay and make the map unplayable.

21-40: Bad. This ranges from just subpar to bordering on criminal. This may or may not affect gameplay, but it is certainly noticeable to all players and can be considered a significant error in the map.

41-60: Average. There is nothing special about the aspect being explored, but nothing very wrong with it either. You are not impressed or disgusted by what you see here. A casual player won't notice anything at all about this category..nothing stands out.

[Freakish]

61-80: Well-done. There are nice touches that distinguish this from most other maps out there. The author obviously put time and thought into adding this to the map. This could potentially be a selling point for the map.

81-100: Excellent. Few maps can surpass what you have here. This is something most players will notice and appreciate. This has only minimal errors (if any) that do not distract or detract players or gameplay in any way.


While the above outline implies reviewers test a map offline against bots, it leaves out an important standing recommendation to all reviewers: play the map against other people if at all possible. As botplay often doesn't uncover gameplay issues encountered online, a reviewer's perspective can be changed drastically by a round or two online. And since many players are looking for maps to play online, neglecting the nuances of network play in a review leaves out information valuable to many gamers.

You may have noticed that the weights for each category are identical to those used in reviews at NaliCity*. This is not a coincidence. We felt that this configuration represented the optimal combination of categories and weightings to describe a map. Any changes to the system would only make it worse. Kudos to NaliCity on choosing such an excellent system.

--UnrealPlayground Review Staff

*NaliCity's schema has since changed.

[blito3]

hey freakish
can u yak with the team
and see if once a review is up
that other can add their two cents
to the review
not just posts
but under the actual review
authors should be excempt of course

[StealthDP]

That feature is coming very soon....

[Freakish]

Originally posted by StealthDP
That feature is coming very soon....

[blito3]

soon?

[Squirrelinabox]

geez blito3, dug this one out of nowhere eh?

[HortonsWho]

Patience-- it's on the way.

[Squirrelinabox]

patience is on the way? what the heck is this patience you speak of and why does it want to come here?