CAP Policy Review: Getting the Bugs Out of Our System

Introduction

Hello again, Smogonites. It's been a while since I've written about CAP for you all. CAP 4 was certainly a roller coaster ride, and in this article I'm going to strap you in and turn your stomach inside out and upside down—but only if you are tall enough to ride.

Maybe that analogy seems a little harsh, but let me try to introduce this another way. Let's say you had a tournament that was run so poorly by one tournament director that it compelled the tournament leaders to create entire threads devoted to the origin and history of Smogon tournaments, their role in Smogon, how they fit into the forums at large, and then proposed that the entire way tournaments are organized and directed be restructured from the ground up. That's about the size of it.

CAP 4 Fallout

So what happened to cause this? As briefly as possible: power creep. Here's a brief history of the power and expectations of the TL.

In the earliest projects, we started with what we called the "mini-mod." Essentially they would guide the project through the process by taking every acceptable entry and putting it up to a vote. This eventually resulted in the "Weak TL" model.

When our community got larger, trying to get consensus by including every legal option became unmanageable. CAP 8 literally had 52 concepts to select from in the first round, and this was long before we implemented the Instant Runoff Vote method. So, after CAP 8 we decided not only should our Topic Leader assess the consensus, they should also help drive and focus it toward competitive ends. This gave us the "Strong TL" model—and our first "Strong" TL, Plus, was very strong indeed!

We eventually relied on our Strong TLs to drive and focus the project so much that they effectively became a literal facsimile of the "mini-mods," right down to giving them actual project mod status. The intent to drive and focus discussion competitively morphed into a way to drive and focus toward the Strong TL's personal goals—more subtly in some cases than others. It was the obviousness of the abuse of that leniency in CAP 4 that brought CAP's entire leadership policy to a head. This series of changes started with the CAP Leadership Compendium Policy Review Thread.

CAP Leadership Compendium

If you ever wanted to know the full story of how CAP came to be and what drives the CAP Forum, read the CAP Leadership Compendium Policy Review Thread.

Most of these Policy Review Threads are topics deserving of an article unto themselves, but the gist of it is that CAP is a separate project from Smogon proper owned by DougJustDoug, and what differentiates us from all the other fakemon projects on the Internet is that we create them as a community for a competitive purpose. The dichotomy this brings is that this is a terrible way to make a competitive Pokémon, but it is relevant and important precisely because that is the way it is created.

Furthermore, CAP projects are huge events. They are essentially a two month-long festival where competitively-minded players duke out their ideas for a new competitive threat in OU, with the end result being a playtest to examine how well the CAP's concept holds out and whether in answering its concept questions we can derive useful competitive data. It is serious because it is fun (our decisions matter and bear out in the playtest), and it is fun because it is serious (the playtest is intended to see how a new threat would affect the metagame and answer questions). Of special note in the event process is the art. Artists and spriters put a ton of work into making the CAP feel like a "real" Pokémon. It's a huge high-stakes contest that regularly draws in hundreds of voters.

The CAP Project is also a natural magnet for accusations of fanboyism and other attacks on whether the project is truly competitive. As such, we need to be very careful with Pokémon-defining moves, stats, and abilities. Aurumoth (CAP 4) is a 600 BST Bug / Psychic Pokémon with the abilities Weak Armor / No Guard / Illusion. In other image-destroying news, Lance Armstrong was last seen gloating on Oprah about using his lawyers to destroy people who accused him of doping. This principle requires that CAP must not operate in a bubble.

Our principle on reviewing our own policies is that no policy that we have created is infallible. We are supposed to constantly re-evaluate our policies based on issues that come up. One of the problems we actually figured out in this policy review section is that we had been breaking our policies on ability polls and their placement for the last three CAP Projects, so what was borne out of a desire to deal with Dream World mechanics morphed into a Flavor Ability poll over the course of several projects that did not have targeted Policy Review sessions in between.

Finally, the entire purpose of the CAP project is about the journey to the results, not the results in and of themselves. The meat of what we learn about competitive battling is done mostly through the arguments during the project, which either do or do not come to fruition during the playtest. The end product is not really the purpose except as a test of the journey to get there. So in the end, the stats, ability, typing etc. are just a means to an end and aren't the focal point of what CAP tries to do. The biggest problem with Aurumoth is our topic leader didn't have that in mind when they were going through the process, and admitted as much afterward.

What the Leadership Compendium did is spell out clearly what CAP is, what it's for, and where it should be going.

Topic Leader and the TL Curse

A topic near and dear to my heart, as someone who took over for Fuzznip during Voodoom (DPP CAP 11) and as the Topic Leader of Mollux (BW CAP 3). DougJustDoug's next Policy Review was on Topic Leadership.

This covered three topics. bugmaniacbob's leadership of CAP 4, the aftermath of that project, and Topic Leadership history. For the first two sections, I recommend clicking the link above and reading—I can't really do it justice and I'm not going to use a Smog article to go in depth about another user's leadership. I've been in that position before. It is not easy, it is very stressful, and there's enough analysis in the thread itself.

What I do want to cover is some of the issues with Topic Leadership. As I mentioned earlier, we went from a "mini-mod" to a "Weak TL" to a "Strong TL." Having been infamous for "hijacking" CAP 8 by exploiting the weaknesses of the "Weak TL" model and having served as a Topic Leader under the "Strong TL" model, it is noteworthy that very few of the "Strong TLs" stuck around after their project was over. reachzero still contributes to a degree, but there was a falling out with Rising_Dusk, Plus is gone, Beej is gone, Fuzznip is gone—basically there was a half-joke that there was a "curse" attached with being Topic Leader. This was discussed in the thread as well, and a lot of it has to do with the pressures of the position.

This brings us to the actual Topic Leadership proposals. The initial proposal was to eliminate the position altogether and break it up into a leadership committee. As this was discussed, it was modified into a hybrid TL/TLT model, with the TLT Members being assigned as slate leaders of a specific competitive facet of the project—those being Typing, Ability, Stats, and Movepool. The TL is assigned to slate the Concepts and manage the Concept Assessment, as well as provide regular feedback and interaction with the slate leaders and take care of the flavor elements at the end. The TL is also entrusted with a veto system of +1/-1/full veto, where they can add an option to a slate, subtract an option, or veto an entire slate, which will then be resolved between the TL and the slate leader and may include moderator intervention.

The eventual outcome was the result of a great deal of discussion. I provided the base model and reasoning for selecting the TLT with those specific positions, Birkal, DougJustDoug, Vader, and the #cap participants provided a lot of feedback to come up with the veto system. The ultimate goal was to preserve the sense of a unified direction that came with the strong TL, continue to give CAP participants a high position to strive for, but to also spread the workload and provide more opportunities for leadership that had actual teeth. In this system, each leadership position has its own real power and checks.

A final thought before moving on. These policies were all done with the intent to fulfill the original vision of the Topic Leader, which is to lead the topics and keep the project on a focused, competitive track. As the community has grown and the members gained more experience, the pressures on the position and the expectations had all risen. As we move forward with the new system on CAP 5, we will have still more work to do to improve the chances for and abilities of leadership.

Recruiting

In some ways a break from all the contentious topics, the Recruiting Policy Review was a much better soundboard for how the CAP forum could do better outreach to the other forums. Theorymon had made a post indicating he would lead the outreach and how to structure that outreach. In general, the thread focused on optics and outreach, which were areas that needed discussion but were not as polarizing.

Abilities

For all the times I've talked about Movepool Discussions having a ton of flaws and being completely chaotic, Ability Discussions have never really been that much better. DougJustDoug's next Policy Review on Abilities went in-depth and made multiple proposals for changes to the process.

Some of these changes were easily done, like a stated ban on custom abilities. In general, CAP has been burned by custom additions before—they are very bad for the forum's image and introduce undesirable elements. Half of the reason Kitsunoh terrorizes the CAP Ladder is Shadowstrike acting as a Ghost STAB, power-buffed Crush Claw. I assure you if Banette or Dusknoir had Shadowstrike they'd be a peg up from where they are, never mind any non-STAB users.

Other insights gained from the thread were that some of our actual established policies had not been followed, and that had led to some of our difficulties—had we just enforced them we would have eliminated a great deal of drama.

One outcome of the Abilities Policy Review was the establishment of a set order for abilities and a series of banlists compatible with that process change. Specifically, after typing there would be a primary ability that would be a defining characteristic of the CAP, then stats would be figured out, and we would decide if a secondary competitive ability was warranted. The banlists established which abilities could not be secondary abilities, with the other lists being entirely banned abilities and purely flavor abilities that could not be discussed in competitive ability discussions.

The other important accomplishment was a ability discussion mechanism that would keep it on track. The result was a Discussed / Not Discussed listing in the thread, much like the one used in the Movepool Discussions to keep them on track.

Flavor Steps

The Flavor Steps Policy Review mostly concerned itself with fixing the meandering way most of the flavor discussions were run. It used some ideas from the Art threads by codifying rules for final submissions and establishing a policy that all legal entries would be slated.

Bringing it All Together

The final Policy Review thread was CAP Leadership Structure, which is still ongoing at the time of this writing. This thread is to work out some of the structure of CAP elements outside the forum like the Battle Server and the #cap IRC channel, as well as summarize this Policy Review period.

If anything, this period has proven that reviewing policies can be a good and necessary thing, and that no matter how good you may think a system is, there is always room for improvement. Policies you use in one part of the system can often be modified for other parts, like Art's submission rules policy on other flavor steps or Movepool's discussion control mechanism on the Ability stage. It also proves that knowing your own history can be very instructive, especially if you have forgotten it as you move forward.

Conclusion

CAP 5 has already begun, with the Topic Leader and Topic Leadership Team already in the process of being selected. We're stepping into both new and familiar territory in the CAP Forum, and we're hoping to have you join us.

Our Policy Review Threads have led to a ton of new resources and understanding of the CAP Process, History, and Policy. We've come to a more complete understanding of the issues inherent in the process, and discussed in great detail the source of some problems and potential ways to resolve them. All of the threads are highly recommended reading, whether you are a long-time fan of CAP or a new user. We will still be moving forward towards a constant improvement in an inherently imperfect process, and hope you will join us.

« Previous Article

Home