Crowd Queue


I live in mild fear of the audience Q&A. What's up with all of those soliloquies masquerading as questions? Also, what about introverts? What if someone doesn't want to line up behind the mic? So I made this free tool to help mix things up. Pass it on.

What does this tool do?

It helps gatherings solicit and prioritize attendee questions et al. People pull out their phones or laptops, type up their questions/ideas and vote on their peers'. Then a sophisticated algorithm ranks the content and everyone is given a list of the "best" entries. Pros: (1) it gets rid of soliloquies masquerading as questions; (2) it provides a safe space for people who otherwise might not ask questions; and (3) it helps moderators actually get to the content an audience wants addressed. Biggest Con: participants must have access to an internet connected device.

How does one use this tool?

If you click "create a new session" on the home page, you'll be walked through the process or running a session. Feel free to create some dummy sessions and play around. A good user interface is like a good joke, if you have to explain it, you've done something wrong. That being said, I've done many things wrong. So I've written a lot of stuff below. Hit me up with questions on twitter @colarusso if that doesn't do the trick. That being said, each session lasts 90 days from creation to oblivion. During that time it has the potential to advance through three stages:

1. Check-In: users check-in with the session code. Users cannot submit or vote on content during this stage.

2. Open Submissions & Voting: anyone who has checked in can submit content and vote on their peer's submissions. New users are not allowed to check-in during this stage.

3. Viewing Results: voting ceases, and content is ranked. Anyone with the session code can view the final ranked ordering.

What do you mean by sophisticated algorithms?

Really, it's not that sophisticated. I just liked the phraseology. Basicly, we're modeling all content as good or bad and trying to figure out how confident we can be that something's good. To that end, an approximate Wilson's score interval is calculated for each entry based on up votes and the total up and down votes. Users work their way through content starting with the least certainly good or bad content. The final list is an ordered list with the highest confidence scores at the top. Consequently, even if participants only vote on a small number of questions, we can say with some confidence that the top questions are actually "good," whatever that means. ;)

What stops someone from voting multiple times?

Nothing can really stop this, but a few things can make it less likely. First, all users have to check-in before voting begins. Once voting starts, there are no new check-ins. Consequently, someone can't simply vote, clear their browser's cookies and vote again. Yes, someone could open multiple browsers during check-in, but since the number of participants is clearly displayed to all users, there is a built-in check on too many people doing this. That is, if the session lists more people than you have in the room, someone has been naughty. Put another way--culture. Culture stops cheating. Think honor code.

What about content that says the same thing in different ways?

It's a simple matter for someone reading a list of questions to filter this out. It's not so simple for computers. See Tasks.

What about silly questions?

Again, this is something a person is better at filtering out, but I've found that people are actually pretty good at taking this sort of voting seriously. So if something is at the top, moderators ignore it at their own risk.

What if attendees don't have smart phones?

This really is a problem, and it's worth noting that this tool isn't always the right one. If your audience doesn't have phones, this is probably a bad tool. If your audience is two people, this is probably a bad tool. I think a simple heuristic would be that this tool is best suited for venues where you'd need a microphone to hear questions or when you want to encourage participation from people who wouldn't otherwise ask questions. It might also be useful as a tool for small break-out groups who could generate ideas for sharing with and evaluation by the group as a whole.

What do you do with my data?

You may have noticed that you don't have to sign in, and I'm not even using Google Anaylitics. So when you say "my data" we're mostly talking about questions/ideas you wrote and your vote history. I don't have any plans to do much with these other than serve up results, but who knows, maybe if this takes off I'll use the questions as a data set to figure out common characteristics of great questions. I am a data scientist after all. I'll tell you this, I'm not going to sell it to anyone.

Who made this?

David Colarusso made this because he listened to this podcast a few weeks before being on some panels at the 2016 ABA TechShow. He then had the tought, "Hey, I don't want to be on a bad panel. I'll build an app."

The tumbs up and arrow buttons you see when voting on content, as well as in the homescreen icon, are the work of Antar and Julien Miclo over at the Noun Project.

GitHub Repo

Create New

Create New