Can I Ask You a Question Please?: A Micro Study

An example of a question that has been scheduled on SocialBee

Question posts are the most popular tweets on my Twitter account. I've never worked on an account where this isn't the case. That alone makes it worthy of devising some sort of test to see if consistently posting questions is a good idea. If I cannot outperform questions consistently, then could I just ask questions?

Question tweets seemed like a great place to start my series of micro studies. And you dear reader, have stumbled upon that very study!

(Notes: This has been peer reviewed for input and ideas. If you want to geek out about the pscyhology of questions, you'll fine more information in the notes section too!)

In this test I asked the various audiences up to 5 questions in a row. I then measured the impact this had on 5 very different test accounts. In this micro study write up we will be reviewing:

  • Likes, Comments and Retweets for the surrounding tweets
  • The quality of responses/metrics for the surrounding tweets
  • The impact on growth over the following 30 days

If you're too lazy to read the entire micro study (who has the time). Then just click the accordion below to read the TL:DR version you scoundrel!

Otherwise... let's begin!

Intro
Hypothesis
Process Notes
What Happened
Conclusions
Notes

The Accounts

In order to protect commercial interests and future studies, I won’t ever reveal all the accounts I have influence over enough to do these tests. However, it’ll be quickly obvious that one of them is my very own twitter account

There are then 4 more:

DD - a poorly performing twitter account that is also very new. It operates in the computer gaming space and in particular the kit and gear that people use for streaming. 

KG - a high performing account in the food business - not a massive following but it is growing!

STA - this is the best performing account in the experiment. Consistently gaining good engagement in the teacher training technology space. 

LT - a cryptocurrency twitter that I have no control over but I had a willing participant - this hopefully adds a little more depth to such a small study!

A table showing the starting positions for all the featured Twitter accounts

Green indicates the account(s) that I believe have indicated they are the strongest for this metric, with red indicating the weaker/weakest accounts.

Hypothesis

You'll be stunned to hear that I do not expect 1 or 2 questions to do any harm at all. Somewhere around 2 or 3 will be the sweet spot. 

Any more will destroy your account! The impact will be sharper than a harpoon that's been especially sharpened!

Some quick predictions:

  • 1 or 2 questions will have a neutral impact with 2 or 3 showing improvements. 
  • Better accounts will be resistant to impact
  • 4 questions in a row is where I imagine every metric will plummet
  • I believe at least 2 or 3 of the accounts will lose followers for this

Process Notes

  • I conducted (almost) the entire experiment from my mobile phone.  This meant collecting data via Google docs. I thought this was an uncommon approach but everyone that commented or reviewed this for me said the same thing, that they always work this way! I’ve been stuck in my Excel ways for way too long it seems!
  • There were 5 Twitter accounts in use and no I won’t share what the other 4 were but:
    • They have varying numbers of followers 
    • The starting point is below
  • Quality was a subjective score. I added this to try and provide a qualitative side to the story! Did I succeed? Who knows...

What Happened

Let’s break this down into 3 key areas where the impact could reasonably be expected to happen.

1 or 2 Questions

Asking 1 or 2 questions in a row is quite a natural thing to do on Twitter. As a result, it wasn’t massively surprising to see that this often resulted in little to no change in the key metrics. 

However, 2 questions was definitely the sweet spot for this test. Every account saw improvements in some, if not all respects. This makes it relatively predictable under the circumstances. This potentially suggests this is in danger of being a real finding!

The outstanding performer on this was the KG account. An account that has a generally low number/quality of responses to their tweets. This would suggest if your engagement sucks then questions may be a good way to get the conversations going. 

Sidenote: I am so glad as I’ve been telling people I know and love otherwise that this is how they get their knowledgeable tweets to show up more too!!!

The results from the top performing twitter account after 1-2 questions

 

The worst results actually appeared for my own Twitter account

Despite this, you’ll notice that there were still positive results from the testing period. Which is why you’ll notice I don’t shy away from asking questions despite this finishing a while ago!

 

The worst performing twitter account after 1 or 2 questions

3 to 5 Questions

Quite predictably, the story changes when you overdo it. This aint Who Wants to be A Millionaire after all!

The biggest loser was STA. An account which generally has a high quality following who are very responsive. 

This experiment absolutely savaged this account! I have to be very careful not to keep overdoing things if I want to see it continue to grow. Slapped wrist for Mr Rice on this one!

One off finding on this account. Was the harm done by asking 3 questions in a row. I cannot fathom why asking 3 questions would harm follower growth so much and more would not! No other clues at all, which suggests it must be a measurement error.

Nobody received a positive enough result during this process for 3-5 questions in a row. Not one! So why bother… 

The results when you ask too many questions on twitter - it is NOT a good idea

Impact on Followers

Even 4 or 5 questions in a row would rarely damage overall follower growth. Which is sort of interesting, but sort of not. I suspect this is because that simply isn’t enough questions to really hack people off enough. Especially with COVID, Boris, Trump and all sorts going down at the moment!

So if you came here to run amock on a social media account you have access to. Then you shall be disappointed to find out you cannot do it in this covert manner!

Conclusions

Asking questions clearly has value. Asking great questions so will result in great conversations too. However, it wouldn’t be a good idea to flood your feed with questions exclusively. 

I would probably look at asking 1 or 2 questions a day at max. 

Pushing it further almost always leads to reducing the result of your work. However my hypothesis that 4 or 5 questions would kill off growth just wasn’t true at all.

As with anything digital marketing - test it, test it, test it.

I would advise you to look at 1 or 2 questions in a row and 2 per day at max. Your ultimate aim is to create a mix of tweets that inspire and add value to your audience. Helping move them towards your goals, whatever those goals are!

If your account has business games. You need each tweet to serve 1 of the following purposes

  • engagement and development of existing audience
  • audience growth
  • adding value to your audiences life
  • clicks to your site

 

That's it!

If you ignore all 4 consistently you will kill off your following! If you want them to know, like and trust you, you'll need to create a better experience for them in future.

Improvements for Future

There are many faults with this micro study but it was definitely worth conducting. It gave me ideas and allowed me to test the hypothesis. However, we've discovered some weaknesses already:

  • Quality score was a welcome addition but needs improving. Maybe this would be better if peers reviewed and agreed these in advance of the study.
  • It would have been good to be able to determine the density of questions vs. other kinds of tweets and see that for comparison
  • It would also be good to see if tweeting too much in general is a bad idea

Notes

Interesting Sources - Yup, turns out I'm a bit of a nerd!

Ramit Sethi - Thought Provoking Questions

Brene Brown - Asked Senior Leaders… 

Dan Ariely - Asking the Right and Wrong Questions

Peer Reviews and Suggestions by

Ruth Barrett
Suganthan Mohanadasan

Leave a Reply