Difference between revisions of "Verification Guidelines"

From SDA Knowledge Base

Jump to: navigation, search
m (wikified)
m (Public vs private verification)
 
(12 intermediate revisions by 5 users not shown)
Line 1: Line 1:
Congratulations on being chosen to verify a speed run. The verification stage of run submission is the first line of defense against cheating and poor quality video.
+
==General==
 +
SDA hosts ”high quality” speedruns. Every run on SDA has gone through a verification by peer reviewers to ensure that it meets the site requirements. The ”high quality” requirement can be broken down into one part gameplay and one part capture quality. The verifiers are also expected to check if the run appears to be legit.
  
==Confidentiality==
 
Please do not give anyone the link to the run or the video files themselves. Furthermore, if the information has not already been made public by the runner, do not tell anyone the final time, what tricks were used, or any other information about the run. Some people prefer not to be spoiled. Finally, do not tell anyone you are verifying the run -- not even the runner. This ensures that your opinions are not swayed by anyone else.
 
  
==Cheating==
+
==Quality (Gameplay)==
Be sure you are up to date on the [http://speeddemosarchive.com/rules.html official rules], and report anything suspicious you see to Mike immediately, either by email, SDA PM, or IRC. The runner will be given the opportunity to explain the situation.
+
While strategies are ever evolving and runs will eventually become outdated, the standards can be controlled at the time of acceptance to the site. Objectively, the ”high quality” criteria for the gameplay can be translated into:
 +
* On par with the fastest runs on the internet for the category
 +
* On par with the fastest runs on the internet for comparable games and categories
 +
 
 +
This can both mean that some World Records should be rejected (typically for games that haven't been optimized enough by the community) and that some non-World Records should be accepted (typically for highly competitive games).
 +
 
 +
If you are not sure what is considered reject-worthy, here are some examples:  
 +
* A death that costs a significant amount of time.  
 +
* A poor, inefficient route and bad planning.
 +
* Using the wrong weapon against enemies or bosses.
 +
* Failing to use the fastest method of movement when it could have easily been used.
 +
* Numerous sloppy minor mistakes (missed jumps, whiffed attacks, failing to perform a shortcut) that cost a significant amount of time (significant depends on the run length and diffculty).  
 +
* Bad RNG costing a significant amount of time
 +
* Not skipping skippable cutscenes (unless the in-game timer is used and not running during cutscenes - it's still very much encouraged to skip cutscenes in this case as well though)
 +
 
 +
Overall, imagine you were in the shoes of the runner. If you think you would have told yourself after the run:<br />
 +
”''Wow, that went really well. There were only a few mistakes, bosses were behaving well and the RNG was mostly favorable – this time will be hard for me to beat without a considerable effort''”,<br />
 +
then it's probably a good fit for SDA!
 +
 
 +
Improvements on existing SDA runs still have to meet the same general criteria, and are thus not given an automatic accept.
  
 
==Quality (Capture)==
 
==Quality (Capture)==
The verification copies of the runs do not usually represent the quality of the final products. However, if you notice any serious problems with the video or audio quality (such as the picture going completely blank or the audio sounding overpeaked, on only one channel or completely missing), or if you notice a missing segment/mismatched segments, please report it to Mike immediately. Also, if there are failed attempts in the video, please tell Mike when the run actually starts and ends.
+
If you notice any serious problems with the video or audio quality, describe the issue and when it occurs in your verification reply (segment and time stamp). Here is a list of some common problems with the audio and video to look out for:
 +
* Audio only on one channel
 +
* The sound is overpeaked
 +
* There is no audio track with only the game audio (it's fine if there is a secondary audio track with commentary, as long as the first audio track only contains the game audio)
 +
* The resolution is wrong
 +
* The run was not recorded at the full framerate
 +
* The video has not been properly de-interlaced (https://kb.speeddemosarchive.com/AviSynth#Part_5:_Deinterlacing_.2F_Full_framerate_video)
 +
* There are skips in the video. It can't be expected that recordings are always perfect, so minor skips are acceptable. It's always good to point them out though to be sure.
 +
* There are failed attempts in the video
 +
* The recording was not a direct feed and contains overlays such as splits, webcam etc. obscuring the game itself.
 +
* An unofficial emulator was used
 +
 +
For additional audio and video requirements, please consult [[Ways_to_get_your_run_rejected_for_video_quality|Ways to get your run rejected for_video_quality]]. Finally, it's not expected that you have to be a professional a/v specialist to verify runs for SDA. Just point out things to the best of your ability.
  
==Quality (Gameplay)==
+
==Cheating==
SDA gameplay standards are high, but mistakes happen. If you feel that the run does not meet SDA standards and that adding the run will not spur others to beat it within a short period of time after it is posted, then tell Mike. The reputation of the site cannot be maintained if sub-par runs are posted. Also, if the run is 100% or low%, make sure it actually meets the definition.
+
Be sure you are up to date on [[Rules|SDA's rules]], and report anything suspicious you may see in the verification reply. Here are some typical examples of things to look for:
 +
* Is there continuity between segments? Between levels?
 +
* Have any cheat codes been used?
 +
* Does the run fulfill the requirements for the category?
 +
* Is there suspicion of non-stock input devices being used (macros, auto-fire etc)?
 +
* Are there signs of segmentation/splicing in a run that's submitted as single segment?
  
Do not feel ashamed if you are rejecting a run.  Ask yourself whether people would really want to see this run.  Sometimes people submit sub-par runs just so they can get their name on SDA.
 
  
If you are not sure what is considered reject-worthy, here is a small list of reject-worthy mistakes:
+
==Public vs private verification==
 +
SDA runs two verification systems in parallel, private and public verification.
  
*A death that costs a significant amount of time.
+
Private verifications are performed in a private PM thread by verifiers having signed up through the ”[https://forum.speeddemosarchive.com/board/speedrun_verification2.html Runs Needing Verifiers]” thread in the verification section of the forum. Verifiers having signed up for a run that ends up in private verification are expected to have specific knowledge about the game. It can for example be from having played through the game, having helped with the planning of speedrunning the game or simply by having gained an understanding of the game from watching speedruns of the game.  
*A poor, inefficient route and bad planning.
+
*Using the wrong weapon against enemies or bosses.
+
*Failing to use the fastest method of movement when it could have easily been used.
+
*Numerous sloppy minor mistakes (missed jumps, whiffed attacks, failing to perform a shortcut) that cost a significant amount of time (significant depends on the run length).
+
  
==Timing==
+
Public verifications are posted in an [https://forum.speeddemosarchive.com/board/public_verification.html open section of the forum]. You don't need any previous experience from the game to verify a run in public verification. It's always a good idea to state in your reply if you have any relevant experience with the game though. You might not be able to evaluate complex puzzle solutions, the routes used, what is RNG and not or know what triggers what in the game. Just base the verification on what you can verify and be open about what you can't check. Here are some examples of what you often can check:
If you are asked to help decide how a run will be timed, give this information to Mike via email or SDA Forum private message (mikwuyma on the forum). Nate has no responsibilities as far as timing goes. If the game needs to be manually timed, please tell us when character control starts and ends. Unless if you are using VirtualDub to get specific start and end frames, DO NOT time the run yourself. If you are using VirtualDub to time the run, then please tell Mike the start and end frames.  If you want to help time a run, but don't know how to use VirtualDub, just ask. Mike likes it when people help time runs, he just wants it done correctly.
+
* Is the movement good?
 +
* Does the runner seem to know what they're doing or are there many hesitations?
 +
* How is the menuing?
 +
* Google [game name] + speedrun and see how the submitted run compares to other runs on the internet (if you know of leaderboards for the game, check them as well)
 +
* Do the run comments explain non-obvious things going on (provided they seem reliable)?
 +
If you are in doubt about something in the run, feel free to ask. Many runners check the verification thread of their run regularly and are often happy to answer questions.
  
==The Importance of Verification==
+
==Final words of advice==
Verification is what separates SDA from YouTube. In the past, there have been very few cases of a run being taken down after being posted due to verifiers missing something, and we'd like to keep that number as low as possible. Thus, we remind you that verifying is a serious task. If you're just looking to watch the newest runs before they're posted, verifying is not for you. When you volunteer to verify, you agree to commit your fullest energy to making sure that when a run is posted, it meets SDA's standards 100%. When a viewer sees the label "VERIFIED: NO CHEATING" at the beginning of a run, we want them to be able to rest easy in the knowledge that it is the truth.
+
While SDA-verification has a clear purpose, please also keep in mind that this is just a hobby and that we're dealing with people on the other end. Keep criticism constructive and don't hesitate to give credit to the runner when credit is due. SDA has been hosting speedruns since 2005 and there is only a handful of examples where verifications got a bit heated before things got sorted out. The vast majority of rejections are taken very well by the runners and in many cases even motivate the runners to come back with an even better run. Let's keep the atmosphere positive also in the future!
  
  
 
Return to the [[Main Page|front page]].
 
Return to the [[Main Page|front page]].

Latest revision as of 10:41, 28 December 2017

General

SDA hosts ”high quality” speedruns. Every run on SDA has gone through a verification by peer reviewers to ensure that it meets the site requirements. The ”high quality” requirement can be broken down into one part gameplay and one part capture quality. The verifiers are also expected to check if the run appears to be legit.


Quality (Gameplay)

While strategies are ever evolving and runs will eventually become outdated, the standards can be controlled at the time of acceptance to the site. Objectively, the ”high quality” criteria for the gameplay can be translated into:

  • On par with the fastest runs on the internet for the category
  • On par with the fastest runs on the internet for comparable games and categories

This can both mean that some World Records should be rejected (typically for games that haven't been optimized enough by the community) and that some non-World Records should be accepted (typically for highly competitive games).

If you are not sure what is considered reject-worthy, here are some examples:

  • A death that costs a significant amount of time.
  • A poor, inefficient route and bad planning.
  • Using the wrong weapon against enemies or bosses.
  • Failing to use the fastest method of movement when it could have easily been used.
  • Numerous sloppy minor mistakes (missed jumps, whiffed attacks, failing to perform a shortcut) that cost a significant amount of time (significant depends on the run length and diffculty).
  • Bad RNG costing a significant amount of time
  • Not skipping skippable cutscenes (unless the in-game timer is used and not running during cutscenes - it's still very much encouraged to skip cutscenes in this case as well though)

Overall, imagine you were in the shoes of the runner. If you think you would have told yourself after the run:
Wow, that went really well. There were only a few mistakes, bosses were behaving well and the RNG was mostly favorable – this time will be hard for me to beat without a considerable effort”,
then it's probably a good fit for SDA!

Improvements on existing SDA runs still have to meet the same general criteria, and are thus not given an automatic accept.

Quality (Capture)

If you notice any serious problems with the video or audio quality, describe the issue and when it occurs in your verification reply (segment and time stamp). Here is a list of some common problems with the audio and video to look out for:

  • Audio only on one channel
  • The sound is overpeaked
  • There is no audio track with only the game audio (it's fine if there is a secondary audio track with commentary, as long as the first audio track only contains the game audio)
  • The resolution is wrong
  • The run was not recorded at the full framerate
  • The video has not been properly de-interlaced (https://kb.speeddemosarchive.com/AviSynth#Part_5:_Deinterlacing_.2F_Full_framerate_video)
  • There are skips in the video. It can't be expected that recordings are always perfect, so minor skips are acceptable. It's always good to point them out though to be sure.
  • There are failed attempts in the video
  • The recording was not a direct feed and contains overlays such as splits, webcam etc. obscuring the game itself.
  • An unofficial emulator was used

For additional audio and video requirements, please consult Ways to get your run rejected for_video_quality. Finally, it's not expected that you have to be a professional a/v specialist to verify runs for SDA. Just point out things to the best of your ability.

Cheating

Be sure you are up to date on SDA's rules, and report anything suspicious you may see in the verification reply. Here are some typical examples of things to look for:

  • Is there continuity between segments? Between levels?
  • Have any cheat codes been used?
  • Does the run fulfill the requirements for the category?
  • Is there suspicion of non-stock input devices being used (macros, auto-fire etc)?
  • Are there signs of segmentation/splicing in a run that's submitted as single segment?


Public vs private verification

SDA runs two verification systems in parallel, private and public verification.

Private verifications are performed in a private PM thread by verifiers having signed up through the ”Runs Needing Verifiers” thread in the verification section of the forum. Verifiers having signed up for a run that ends up in private verification are expected to have specific knowledge about the game. It can for example be from having played through the game, having helped with the planning of speedrunning the game or simply by having gained an understanding of the game from watching speedruns of the game.

Public verifications are posted in an open section of the forum. You don't need any previous experience from the game to verify a run in public verification. It's always a good idea to state in your reply if you have any relevant experience with the game though. You might not be able to evaluate complex puzzle solutions, the routes used, what is RNG and not or know what triggers what in the game. Just base the verification on what you can verify and be open about what you can't check. Here are some examples of what you often can check:

  • Is the movement good?
  • Does the runner seem to know what they're doing or are there many hesitations?
  • How is the menuing?
  • Google [game name] + speedrun and see how the submitted run compares to other runs on the internet (if you know of leaderboards for the game, check them as well)
  • Do the run comments explain non-obvious things going on (provided they seem reliable)?

If you are in doubt about something in the run, feel free to ask. Many runners check the verification thread of their run regularly and are often happy to answer questions.

Final words of advice

While SDA-verification has a clear purpose, please also keep in mind that this is just a hobby and that we're dealing with people on the other end. Keep criticism constructive and don't hesitate to give credit to the runner when credit is due. SDA has been hosting speedruns since 2005 and there is only a handful of examples where verifications got a bit heated before things got sorted out. The vast majority of rejections are taken very well by the runners and in many cases even motivate the runners to come back with an even better run. Let's keep the atmosphere positive also in the future!


Return to the front page.

Personal tools