This rubric is going to talk briefly about what makes a review useful and how you can write reviews that help your fellow judges grow.
Review writing was covered in some amount of detail in the Level 2 Lesson, but this elective is focusing on how we evaluate reviews for the L2 Mentor role.
We’re looking broadly at five categories: Actionable advice, accurate evaluation, constructive criticism, meaningful praise, and overall tone. Each category is rated deficient, adequate, or exceptional.
Actionable Advice
Actionable advice is perhaps one of the most important parts of the review - giving the subject of your review a way forward and the tools to improve is critical to their long-term success, rather than just telling them that they’re good or bad.
Deficient
Deficient advice will be non-specific at best, non-existent, or inaccurate at worst. It can also be advice that is not something that the candidate can reasonably follow through on.
Example: “Your understanding of nested abilities is bad, and you should study more.”
Example: “Your understanding of how triggers resolve is poor. I recommend reading the Magic and Lorcana comprehensive rules on triggers, as they are functionally similar.”
Example: “You need more large event experience. You should travel to the Sector Qualifiers in both Europe and the United States to get it.”
(The author would like to note that the way Magic and Lorcana triggers work is meaningfully different from how triggers work in Star Wars™: Unlimited)
Adequate
Adequate advice will point the subject in the direction of helpful resources.
Example: “Your understanding of how triggers resolve is poor. I recommend reviewing L1 Lesson 9 and L2 Lesson 3.”
Exceptional
Exceptional advice will be customized to the specific subject, rather than generic, useful advice.
Example: “As we discussed, you sometimes still struggle with nested abilities. In addition to L1 Lesson 9 and L2 Lesson 3, there are some other ways you can think about it. It may help you to physically move the cards - whenever something triggers, put it beneath the event that triggered it, but above everything else waiting to resolve. Then, when the event resolves, you’ll see that Krayt Dragon is there, and it can act as a reminder that it needs to resolve next.”
Example: “You seem a bit uncomfortable taking calls. I recommend asking your friends to call you over when playing casually - that way you can get some experience and develop comfort in a low-stress way. Next time we play, I’ll make sure to call you over and ask you some questions about Ezra and Rebel Assault.”
Accurate Evaluation
To give useful advice, you need to fit it to the candidate. Misunderstanding the subject of your feedback leads to giving mismatched feedback to the candidate. An L1 candidate believing that Thrawn - ... How Unfortunate duplicates Bounties is incorrect, but understandable. A Sector Qualifier Head Judge making the same error is much more significant.
Deficient
Deficient evaluation will either draw incorrect conclusions or show a misunderstanding of the expectations of a judge at a certain level.
Example: “While you were proficient at running a prerelease, you struggled with deck checks and didn’t understand how to run a top eight effectively. This makes you unprepared to be a level one judge.”
Example: “When the player Darth Vader asked you about how On Attack triggers worked, you didn’t respond confidently. You need to focus on your understanding of triggers and their resolution.”
Adequate
Adequate feedback is correctly matched to the candidate and expectations for the level that they are at, or the level they are seeking, as appropriate.
Example: “You seemed nervous making your announcements, but that’s understandable given that it was your first event, and you managed to get the important information across to the players. I don’t have any concerns for you acting as a Head Judge for weekly play.”
Exceptional
Exception evaluation not only accurately compares the candidate to expectations and standards for their place in the program, but also within the context of their journey. In some cases, the evaluation will also include an underlying insight into why the candidate behaves in this way.
Example: “Luke, you’re comfortable making announcements to a group of about eight to sixteen players, but when the group gets above twenty, you start to struggle. While your store usually only gets ten to twelve players, if there is ever a popular prerelease, I am concerned that you might feel overwhelmed. However, you’re able to meet the needs of your community, so I think you’ll be a good fit, as long as you continue to put in some effort and become comfortable addressing a larger group.”
Example: “You’ve mentioned players sometimes ignore your authority and resume playing before you finish your ruling. You have a tendency to overexplain your rulings. This is a trend common in newer judges, because it is information that is exciting to you, and you enjoy the minutiae. Most players don’t need to know why Chirrut or Lurking TIE Phantom survives; they just need to know if it’s defeated or not. Asking “Do you want the short explanation or the long?” may help avoid some of the awkward situations where the players are ready to move on, and you’re not.”
Constructive Criticism
Being able to highlight areas of improvement and speak of them in a constructive way that makes the subject comfortable in accepting the feedback is crucial. Tying back in to actionable advice - providing a path forward is an important part of this process. Simply telling someone that they’re bad at a thing isn’t particularly helpful.
Deficient
A reviewer who is deficient in providing constructive feedback may not give any critical feedback at all, or may cushion it with enough praise that it renders it almost meaningless. On the other end of the spectrum, they may be too harsh in their feedback and not be constructive at all.
Example: “Jar Jar, it was great working with you for prerelease. Although you gave away every card the store owned, and it now has to shut down, you were a pleasure to work with, and I’d happily recommend you to other stores in the future.”
Example: “Savage, you’ve repeatedly shown that you have no respect for the needs of other members of the team, and frequently interrupt judges taking calls, and take over for them. You are exactly the type of person that made me want to be a judge, so I could root out scum like you. You should quit judging. You never should have even tried.”
Example: “You’re bad at policy. I don’t think you should answer calls other than rules calls.”
Adequate
Adequate constructive criticism brings up the issue and addresses it directly, but not harshly. They provide some steps for improvement, or at the very least acknowledge that the evaluator doesn’t have actionable feedback.
Example: “You seemed to really struggle with understanding which penalties to issue when we were working on deck checks. Reviewing the documents and lessons again might help, but what worked for me was more exposure and working with judges with more experience than either of us.”
Example: “You wanted to use deck lists and perform deck checks for your prerelease. That’s not something that is done because it creates a much more intense experience for new players, and is a bit less welcoming. I think you should find someone to have a conversation about your expectations for more casual events, and your fear of players constantly cheating.”
Exceptional
Exceptional constructive criticism lays out a specific path and ways that the candidate can improve or measure themselves against, in addition to being direct and compassionate.
Example: “While performing a deck check, it took you roughly twelve minutes to sort and verify a deck, which is significantly longer than the expected average for an L2 candidate. While developing further familiarity with the cards and the metagame is helpful, I think what will help you the most is practice. Next time we’re both at Tosche Station, I can bring some decks and decklists, and we can get practice sorting. I bet that if you get the reps in you can easily get it down to nine minutes, if not eight.”
Meaningful Praise
In addition to being able to critique other judges, highlighting things that they do well is important to encourage judges to continue that behavior. However, it can often be difficult to find something specific to highlight or call out. When praise is vague, it can often feel less genuine.
Deficient
Deficient praise is often exceptionally vague, so that the subject doesn’t have knowledge of what behaviors in particular are praiseworthy. It can also be absent entirely or backhanded.
Example: “You were a pleasure to work with. You really know your stuff!”
Example: “I was impressed with your rules knowledge, for a level one judge.”
Adequate
Adequate praise points out strengths and either highlights why you believe the subject may not be aware of the strength or why it caught the reviewer’s attention.
Example: “Anakin - you have a confidence that I think comes naturally to you, and that not many other people have. Your ability to be firm in your rulings, even when challenged, is great.”
Example: “Obi-Wan, you mentioned that you like to try to speak to everyone on staff, and I saw that you did! I see many who don’t and I just wanted to say that I appreciated the conversation we had.”
Exceptional
Exceptional praise goes into detail so that the subject can replicate their success, while remaining genuine.
Example: “Mace - the way you explained your thought process during the debriefing was very helpful. I’ve been struggling with understanding how to investigate, and your step-by-step breakdown was something that was very meaningful to me. In particular, the phrase “both players can be telling the truth and also both be wrong about reality” is something that struck home, and is something that I knew to be true, but didn’t have in words. Now I’ll be able to use that language with players while I’m trying to resolve complex board states.”
Overall Tone
This last is more generic and not specific to any piece of content. The judge program is a community of growth and acceptance. This is definitely the vaguest of the five criteria, but it should be relatively easy to meet. It is important to be polite and respectful of our fellow judges and community members.
There is a time and place for light mockery and having fun at each other’s expense, but it’s only appropriate if the subject is on board and an enthusiastic participant in the mockery. The examples focus primarily on critical feedback, as that is the area where tone has the most impact.
Deficient
The evaluator is unkind or disrespectful toward the subject.
Example: “Anakin, while our boss says I have to work with you, I’m not going to endorse you for L1.”
Example: “Honestly, you kinda suck at giving rulings. I don’t know what to tell you, it just comes naturally to me.”
Adequate
At a minimum, an evaluator should be professional when delivering critical feedback.
Example: “C-3PO, I’ve watched you give many rulings, and frequently there seems to be some confusion or uncertainty on the part of the players. Your rulings are technically correct, but your delivery could be improved.”
Exceptional
The evaluator is able to be supportive and personable, even when delivering critical feedback.
Example: “Hey Ahsoka - I know you’re worried that people are only following you because of your rank, and not because you’ve earned your leadership role. Frankly, that’s probably true - but that’s okay - that’s what happens to everyone who is new to a leadership role, unless they have exceptional presence, like a certain Skyguy we know. It’s something that I went through myself and there are a few things that I tried that worked, and a few that didn’t. First...”
These examples aren’t the only way to write a review, and certainly not the only way to provide feedback. If you’re a Level Two judge and are interested in applying for the Mentor role, so that you can endorse L1 candidates, you can navigate to your profile, select “View My Applications” and then select the Level 2 Mentor role.
That’s all for this one! As always, good luck and have fun!