Evaluation & Mentorship of Judges
Aspects of Judging
Also available are archives of live broadcasts, where the Program Director goes over the lesson, answers any questions that folks may have and sometimes goes on tangets about other elements of judging. You can find the playlist of broadcasts on youtube.
Click for Translation
While proper translation and localization are among our long-term goals, we are currently offering Google Translate on the page. Please keep in mind that the translation is automatic, which means that specific game terms, names of cards or mechanics, or technical language used to describe the game may not translate well. As with the documents for the game itself, the English page is the authoritative document in case of any confusion or discrepancy.Join the discussion in our Discord and talk with other judges about what you learned, and ask any questions you may have!
Hello there!
Welcome to the Aspects of Judging lessons. As always, I’m your host Jonah, and this elective lesson series is going to cover the Aspects of Judging - just like it says on the box.
While the last lesson focused on how you can grow and improve personally, this lesson is going to be dedicated to helping others learn, as we discuss the Evaluation & Mentorship of Judges.
A huge part of being a judge is being an educator - on the basic level, in most calls, we’re educating players on how the game works or how we can repair a disrupted game state. Beyond that though, as a judge you’ll be interacting with other judges or judge candidates, and being able to help them grow and improve is something that will help keep the community flourishing over the years.
As judges come and go, having bonds with others and knowing that you can rely on them to help you and to have your best interests at heart means that you can feel comfortable taking on the challenge to improve yourself.
Even as an L1 or apprentice at your LGS, you’ll still be in a position to mentor people. While you may not work with many certified judges, it’s not uncommon for a player to be interested in judging and ask how they can get into it. While the online resources and lessons are a good starting point for certification, a judge who is given individual mentorship and guidance from another judge is so much more likely to learn from their mistakes, stay invested in the community, and be able to give back.
We’ll go over each of the sub-categories and the details within them, discuss what they mean, talk about how you can improve them in a broad sense of the skill, and outline the expectations for judge levels one, two, and three.
Development of other Judges
This one is what it says on the tin - helping other judges get better. It can be through objective feedback and guidance, such as explaining how the rules work, or it can be subjective, such as how you might word your announcements differently.
Creating New Judges
The first avenue is in creating new judges, or helping judges who have less experience than you. When you’re working with individuals who have less experience than you, you’re going to have to change your framework - some things that are obvious to you might not be obvious to them. You might also have to cushion some critical feedback if they’re not used to or comfortable with an environment of honest critique.
Helping a judge earn their first certification (or an advancement certification) is an exceptionally rewarding experience, but being someone’s mentor can also feel like a lot. Making sure to have reasonable boundaries is key, but you can decide what you find reasonable.
Giving Feedback to Authorities
While helping newer judges is more approachable because identifying areas of improvement is easier and their strengths haven’t been as refined, it’s also important to help judges who have seniority or have more authority at a given event than you.
No judge is perfect, and being able to tell your Team Lead or Head Judge how you think they could improve or what they did well is incredibly valuable - as judges become more senior, fewer and fewer people are comfortable giving them feedback or advice, which is unfortunate because they still have places where they can grow.
You can even give feedback as a player to a judge - while they have authority over you in the event, you can still praise clear and concise announcements, or have a conversation about how you might have handled a complex call that happened at your table. Of course, “it’s feedback” isn’t a reason to be any less respectful and it can be a delicate act in providing critique to a judge for an event that you’re actively in, as it may seem like you’re arguing for a different ruling or questioning their authority.
Developing Peers
Beyond teaching judges who have less experience than you or pointing out how judges with more experience could improve, there are all the judges who are around your level - these can be some of the easiest folks to give feedback to because they’re learning the same lessons you are, just potentially in a different sequence given what roles they have at events. It can also be difficult because the people around your level are often the people who you spend the most time with and are the people you’re closest with, and critiquing your friends can be challenging.
However, it’s important to avoid echo chambers. While hyping up your friends, particularly when they succeed, has an outsized impact on morale, doing so without consideration of their areas of improvement or without being honest about them means that they’ll begin to develop a sense of inviolability and may become less susceptible to critical feedback, which will hamper their longer-term growth.
Teaching Hard Skills
Teaching hard skills, objective and measurable knowledge is a fundamental trait of judges. You need to be able to explain the rules to a player. Without that, you’re lacking in one of the key areas of judging behavior - see the whole Game & Format Knowledge aspect. Taking it one step further to teaching judges those same skills is huge. It does require a bit more in depth knowledge - usually one step further. Being able to answer “why” questions, and particularly understanding what happens to the game if it doesn’t work as written, or to point out examples where the rule is relevant helps ground your explanations.
Teaching Soft Skills
Of course, as you go through these lessons on the aspects, you’ll realize that there’s a lot more to judging than the Comprehensive Documents and what they say. There’s a lot to understand about how judges approach calls, interact with players and other judges and logistics and manage the many multivariable challenges that arise during an event.
It can be a lot harder to teach these more subjective skills, as it is more of a matter of perception and best practices, rather than hard and fast rules. A strategy that works for making one person comfortable making announcements may totally derail someone else. Having an explicit checklist for how you handle tasks may keep you on track but simply be confining to another judge.
This skill is all about learning what works for different types of people, and understanding how to provide support in the way that is most beneficial for your student.
One-Shot Mentorship
Sometimes you’re going to work with someone once, or you have very limited interactions and so you only have a tiny piece of feedback. Figuring out how to deliver feedback in a concise and meaningful way from a small interaction allows you to become a powerhouse on the floor of an event, whether as a player or a member of the judge staff.
This can be feedback as small as your perception of an individual call, or something larger like an event weekend.
Legacy Mentorship
On the other hand, sometimes you’re close friends with somebody and go on road trips to events together. You work with them every few weeks or months, and have spent time building up a rapport. You will have a better understanding of how they think, what works for them, and what doesn’t, but it also means that you can tackle more complex areas, such as a discomfort with being in a position of authority, and work with them over the course of many events, rather than a quick hit. Long-term mentorship is frequently more aligned with subjective skills, and one-shot mentorship is more frequently about rules and policy, but those aren’t hard and fast rules.
Delivering Critical Feedback
Being able to point out flaws or criticisms and have that feedback not only be heard, but appreciated and responded to is not easy. Even open minded folks who strive to learn and accept critical feedback will often reflexively dodge critical feedback, and some of the most experienced judges who do seek out this critical feedback will be able to rationalize their actions.
Being able to explain your feedback and your perspective, and underline why you think it’s important, to get your subject to appreciate it and re-evaluate their stance is huge. Critical feedback doesn’t have to be negative feedback - it can be constructive and reinforcing, and a generally positive experience, even if it lays bare more difficult topics.
Actionable Feedback
Of course, another piece is giving the subject of your feedback a direction, a place to go. If all your feedback does is highlight areas that are strengths or weaknesses, but without advice or direction, it can fall flat. While there are times when the best feedback you can deliver is “this was good.” or “this was bad.” but expanding a bit further with “and here’s why” adds another dimension that allows the subject to identify those areas on their own in the future.
Adding “and this is what you can do in the future” adds yet another dimension that allows them to try to overcome the hurdle or reinforce their good behaviors, which is what actually leads to change. Making it so the subject has to figure out what needs to be altered adds a significant barrier to overcome before progress can be made, and so by providing the avenue, you help get the subject to a state where they’re more likely to be able to receive and act on your feedback.
Level One
At level one there isn’t a significant expectation of mentorship of judges. You have the fundamental knowledge to educate players, but you aren’t trained or tested in developing judges. However, something that you have complete knowledge of is the process of becoming a certified judge. Being able to guide someone else through the process of certification makes for most of the expectation. Being able to answer rules questions of a judge candidate is above and beyond, but appreciated.
Level Two
At level two, there is a dedicated bonus certification that allows you to endorse L1 candidates. Being able to provide guidance on the certification process through L2, as well as some amount of feedback to candidates for L1 and L2 is generally the expectation.
Level Three
At level three, you’re starting to step into a role of intrinsic leadership within the broader community. There’s a more explicit expectation to be able to provide concrete mentorship and feedback to judge candidates, and you’ll be generally expected to provide some feedback to the judges under your authority at events with multiple judges.
Improving Your Development of Other Judges
This is something that comes through practice - however, like with some of Interpersonal Skills & Communication, this is something that you can get immediate feedback on. You can lampshade your uncertainty and ask for feedback on your feedback.
You can try different formats (flash feedback on notecards, lengthier reviews, or sit-down conversations to name a few) to see what works best with your style. You can focus on building up your peers or finding ways to help people overcome challenges they face. You can try to identify areas that the subject isn’t aware of or focus on actionable feedback for something that is already known.
While is this is yet another area that falls into “just get out there and do it”, you can get a response from the subject immediately and adapt more dynamically than you can in other areas that are dependent on experience.
Evaluation of Other Judges
Part of being able to provide good feedback and help send someone in the right direction is to know what feedback is useful for them. Telling the Head Judge of a major destination event how damage persists probably isn’t helpful, but explaining it to a judge candidate who hasn’t even played a game yet might be more useful.
Similarly, digging into the complexities of a card counting investigation with a Head Judge to try to figure out how the two of you could have been a bit more efficient is probably super valuable - it’s not the sort of call that the Head Judge is going to have a lot of opportunities to share with other judges. On the other hand, sharing it with the aforementioned judge candidate is just going to be overwhelming.
Knowing what tone to strike, what areas to focus in, what strengths and weaknesses the candidate has... identifying these sets you up for feedback that is impactful and lasting.
Accurate Understanding
One part of this is just understanding where the candidate is. Knowing what they know, and more challengingly, knowing what they don’t know. Identifying gaps in knowledge is really hard to do on a personal level, but it’s a bit easier to identify for someone else.
Furthermore, you want to evaluate them in an appropriate context - and that often means in the context of their goals. If someone comes to you and asks for feedback because they’re interested in being a team lead, it’s important that your feedback is on that level. If they were a great floor judge, but not quite ready for leadership, telling them that they were fantastic is misleading and harmful to their growth, because it likely means that they’ll try to overshoot.
Paired Mentorship
Part of evaluating other judges (and this will tie into logistics a bit), is pairing judges up. If you have someone who is interested in being a team lead, pairing them up with someone who is doing the task for the first time gives them the opportunity to practice their leadership and find out how well they can give instructions. It also gives the less experienced judge a partner who is excited about mentoring them and providing support, so that both judges have a more positive experience.
Understanding who can benefit from the knowledge and wisdom of whom else can create stronger scenarios for everyone involved.
Level One
At level one, the expectation is little more than a vibe check. Do you think that a judge candidate would be a generally good fit as a judge? Does someone with more experience seem to fit into their leadership role? Being able to provide more concrete analysis is always appreciated, but not expected of the more individualistic and store-focused judges.
Level Two
At level two, you begin to be able to write endorsements. Being able to put into more explicit words why you think someone is a good fit for certification becomes more important. Furthermore, as you work with more judges, being able to identify where your teammates are and what growth opportunities would be good for them becomes valuable - even if you’re not in an explicit leadership position, being able to provide support as a peer is great!
Level Three
At level three, you’re writing more endorsements and will generally be providing more feedback. As you grow in the program, your understanding of benchmarks should grow, and you’ll have more examples of judges at each certification level, as well as examples of judges who specialize in various areas. As your sample size grows, you’ll be able to more accurately identify judges who are close to (metaphorically, not procedurally) leveling up, and what it will take to get them there.
Improving Your Evaluation of Other Judges
The answer to how you improve this skill is more talking! Talk with other judges and see if your evaluation of them lines up with their evaluation of themselves. Talk with your peers about each other (not in a gossipy way, but in a collaborative way), and see if your understanding lines up with other perspectives. However, evaluation is relatively subjective, and just because you may have a different perception than your peers, it doesn’t necessarily follow that you’re wrong or that this skill is a weakness - it might just mean you’ve seen something they haven’t, or the inverse.
Mentorship Logistics
Sometimes mentorship will follow you into the planning stages of your work, or how an event comes together. Developing teams, or working on schedules, are the most obvious ways you can see this happen, but it’s a combination of more specific individual mentorship as well as Leadership, Management, & Command.
This is an almost passive sort of mentorship - setting up circumstances where judges under your authority can find growth, without you (or anyone else) having to provide explicit feedback.
Team Development
Team Development isn’t just paired mentorship, but it’s looking at how you can grow your team as a whole. It’s about balancing new hands-on experiences with reliability for event performance. It’s making sure that, as a team, everyone is still having a good time and coming away from the event with their goals met.
This is more explicit with judge teams, as judges will have more time during an event to talk to each other, but as an LGS judge, your team includes the store staff and the players - it’s really a stand-in for “micro-community.”
Making sure that people are able to grow from the opportunities that the event gives them is a great way to ensure that those people have a good time and feel motivated to return - whether it’s as a member of staff or as a player.
Building a Team Schedule
This subskill is definitely more skewed towards higher level judges who will be acting as a head judge or team lead. Balancing the resources that you have. Experienced rocks you can lean on, reaches who you trust, but haven’t yet had the opportunity to solidly prove themselves, and new hands who are unknown to you need to be distributed across your event to ensure that everything runs smoothly.
However, putting all of your rockstars handling logistics means that the other judges won’t have the opportunity to gain that experience, and therefore if you lose one of the experts at a future event, you won’t have trained up a potential replacement. Making sure that the staffing assignments set you up for success in developing your teams is key.
Level One
At level one, you’re generally not going to be managing any other judges or staff, but knowing the goals of the people participating in your events, whether they’re store staff or even players, allows you to create an event experience that’s in line with what they’re looking for.
Level Two
At level two, understanding the dynamics of the team, and the importance of balancing your team is expected. It’s likely that newly promoted judges will pay it forward and offer opportunities to more reaches than is necessarily wise, but that’s also expected.
Level Three
By the time a judge is level three, they’ll be a bit more balanced in creating team structures and will have the ability to identify what structures and assignments will result in the best experiences for judges.
Improving Your Mentorship Logistics
While the best place to get more comfortable with this skill is through performing it, a good backup is just being on teams, and trying to parse why the schedule is the way it is, or why judges were given the assignments they were given. Now, sometimes the answer is “they asked for it” or “nobody asked for anything, and so the HJ through names on darts”, and so it won’t be perfect, but it can help exercise those muscles.
Program Structure & Systems
This is a broad scope sub-aspect. It’s about mentorship on a more fundamental level, as well as on a wider level than individual feedback. Part of it ties into the evaluation of judges, as it’s about understanding the program as a whole, but it’s also about being able to help the program grow, and not just individual judges.
Sometimes that growth is through feedback to Cascade Games, and other times that growth is through the development of many individuals.
Benchmarks of Knowledge for Each Level
One of the first steps is knowing what the benchmarks of knowledge are for each level. It’s obviously important to know what the benchmarks are for the certification you’re seeking so that you know what your goals should be, but it’s also fundamental in helping judges advance.
If someone wants to be a level three judge, but also doesn’t really want to get involved in understanding competitive policy, that’s a mismatch of understanding the benchmarks of level three.
Goals of the Program & Levels
Tied into that are the goals of the program and the goals of each level. The judge program is designed to help support organized play in a generic way - L1s are focused on the more casual in-store play, L2s are still mostly store-focused but with an understanding of competitive play, and things evolve from there - but each certification has its purpose.
Knowing what the purpose of each level is, and why it has the prerequisites that it does, as well as the more general expectations (as laid out in these aspect articles or in various other lessons), will help you and the candidates you’re working with find the best certification for them. If someone doesn’t want to leave their LGS, L3 probably isn’t the certification for them.
Flaws With the Program
The program isn’t perfect - some of the flaws are known, but accepted as costs of hitting other goals and benchmarks. Others are known, but no solution has arisen. Others just aren’t seen, because even with all of the experience that the Cascade Games team has, we still have a limited perspective. Being able to provide feedback or to understand that there are areas that need to be worked around allows candidates to have a smoother process.
While the program strives to increase accessibility, there will always be hurdles where the resources required to make the process better simply aren’t available, and so dealing with some clunkiness is to be expected.
Being able to guide someone through those hurdles of certification is one of the more practical tools to being a valuable mentor. They are elements that can feel like invisible walls, which can be erased with just a bit of knowledge or experience... but it can be difficult to find that information.
Expectations for All Levels
At all levels, you’re really just expected to know what’s expected of your levels, and the levels you’ve passed through. If you’re interested in advancing, having a concept of the expectations of the next certification is critical, but you’re not expected to be a master until you’ve actually had time to experience the role.
Improving Your Understanding of Program Structure & Systems
These Aspect lessons are one of the best ways to understand what the judge program is trying to do on a broad scale. Similarly, the introductory lessons for each certification of “What is [Certification Title]” are great for setting up expectations for the various certifications.
This one is challenging, though, because part of it is “do you understand the thought process of the publisher and the designer of the program,” which is sometimes “can you read minds,” which is not a common trait.
However, as the program grows, there will be more and more documentation, and you can always reach out directly to those involved in the program with questions or when seeking clarification.
With that, we wrap up Evaluation & Mentorship of Judges. At least, it’s enough to establish a baseline we can build from. Next lesson will be our sixth and final aspect lesson, covering Leadership, Management, & Command, where we’ll get to put some of these lessons in mentorship into action.
If you’re watching this on YouTube, and you want more lessons in your feed, go ahead and subscribe. Join us after new lessons on twitch.tv/swu_judges for live broadcasts covering the content of these lessons as they are released, and join the Star Wars: Unlimited Judge Program Discord to join the community in discussion of this and much, much more.
Until next time, good luck and have fun!