Your Survey Is Lying to You

For a typical state society, the annual conference represents more than half the operating budget. Most associations still design that agenda off a member survey. Talkadot's Association Event Design Playbook found the survey is quietly lying to you. Not because members are dishonest. Because the format is wrong.

I am Arel Moodie, cofounder of Talkadot. Talkadot is a platform that helps event planners find and book professional speakers using real audience feedback data, and helps speakers capture audience feedback, testimonials, and leads through a simple QR code. I spent a day at the ALC Summer Conference with about 20 state society Executive Directors, per Talkadot's own count on the published playbook. Those EDs represent the Association Leadership Coalition, which collectively serves more than 4,000 associations and 14,000 association professionals across 35-plus state and international societies, so the pattern in this article is not one association's problem. One thread ran through every conversation: the survey everyone runs is the reason the agenda keeps landing flat. This builds on the four-quadrant framework behind the playbook, so if you have not read that one yet, start there.

What Members Actually Do With a Survey

Ask "what topics do you want?" and you get answers that sound right. Networking. Education. Professional development. Then you build the agenda around those words, and attendance still slips, and nobody can explain why.

Here is what is actually happening. Members are honest about their frustrations. They are not honest, and often cannot be, about what would fix them.

They know exactly what is not working. They rarely know the diagnosis. They almost never know the intervention. So when you ask "what topics do you want," you are not collecting a fix. You are collecting a guess dressed up as feedback.

Stop Asking for Topics, Start Asking for Frustrations

The fix from the playbook is a four-step method: Frustration, then Diagnosis, then Skill, then Application.

Members own step one. They will tell you plainly what is not working. Board members who will not lead. Emails nobody reads. Registrations that trickle in at the last minute. That is real signal.

Diagnosis is your job, and your experts' job. Why is it actually happening. "Nobody reads our emails" is rarely an email problem. It is usually a communication-architecture problem: too many channels, no single one members actually check.

Skill is the capability that closes the gap. Application is whether attendees can practice it in the room, not just hear about it.

Run your next member-research question through this filter: "What specific part of your job feels harder than it should right now?" That question collects a problem. "What topics do you want?" collects a title.

Point to It on the Agenda

Here is the test the playbook uses for everything: can you point to the specific place on your agenda where a real member frustration gets solved? Not "we value professional development." The actual session, the actual moment.

If you cannot point to it, you have not designed it. You built a title, not a fix.

With more than half your budget riding on this one event, that is not a small gap. That is the difference between a conference members tolerate and one they plan their year around.

A Free Alternative to the Annual Survey

The playbook has a second tool worth running alongside the frustration method, and it costs nothing to try at your next event. Ask any attendee, or build it into check-in: "Is this your first time attending, or have you attended before?" If they are new, follow with "What made you show up?" If they are returning, ask "What makes you come back?"

Two rules make it work. Ask "what," never "why." "Why did you come" makes people defend themselves. "What made you show up" invites a story. And listen for the answer's value stage: "I love the people here" points you toward belonging, "I always learn something I can use" points you toward content, "I want to see where the industry is headed" points you toward strategy. Whatever they say, that is where you take the conversation next.

Once you have real frustrations and real answers to the magic question, the next place to use them is your speaker and session selection process. We cover exactly how to rebuild that in Your RFP Process Is Approving the Wrong Sessions Right Now.

The full method is in Talkadot's Association Event Design Playbook, along with worked examples from real Executive Director frustrations: the board member who tries to do everything, the email nobody reads, the last-minute registrants. Get the free 43-page playbook and the one-page framework at talkadot.com/playbook.