Comment System Survey Results | Explore

by Candice Rusch, Director of New Media at explore.org

Thank you to everyone who participated in the survey and gave us feedback about what you want to see in a new comment system. In this blog post, I’ll lay out our goals and analyze the results. Before we begin, I want to offer two caveats. We are still in the planning phase of development. Things can, and likely will, change. The features outlined here may not be feasible. Or, we might need to release these features in phases. The primary goal of the survey was to help us understand what features to prioritize in development. This new system will be for the community and we need you to be a part of this process with us. The second caveat is that we removed feedback from the analysis that was specifically related to moderation policies. This survey was designed to help us understand how to better design a comment system. Moderation policy is a different conversation. Don’t worry, we did read your feedback, but if it was related to moderation then it is not reflected in this report.

Stability

It is no secret that Disqus has been buggy. 21% of the free answer portion of the survey was people lamenting how buggy Disqus is as a system. Most of our feedback inbox is filled weekly with people who face issues with the Disqus comment system. Snapshots don’t post. Comments are inaccurately marked as spam. Actual spam floods the site. Comments disappear. Notifications don’t work. The list goes on. The reason we periodically create new comment boards is that Disqus gets even more buggy as a comment board fills up. 

Our primary goal is a new comment system that is stable. To accomplish this, we will have an extensive beta-test. During this test, the new comment board will be placed on a handful of cameras. We will need you to help us test this system: find bugs, tell us where things didn’t function as anticipated. The more feedback the better. I cannot say that our new system will never have bugs, but instead of having to wait days and sometimes weeks for a resolution from Disqus, we will be able to address the issues ourselves. Having our own system also gives us the opportunity to be  transparent with you about fixes and their implementation. 

Features That Fit Our Needs

Disqus is a 3rd party system that we license. We don’t have any control over what features they release and when. This has caused some issues in the past. Most recently their “advanced moderation” feature catches legitimate comments in a spam filter, and we have no ability to opt-out. Even worse, we don’t have the ability to add features that would benefit our community. We feel that the recent tensions around the off-topic rule is a design problem, not a policy problem. This survey revealed that for every person who was sad about the off-topic rule enforcement, there was another person who was happy that the chats are focused on the cams. The community is split in half. We are trying to please two diametrically opposed groups with a system that doesn’t allow us to make any accommodations. Thankfully, you also gave us some great ideas about how to create a chat that can fit more casual conversation, while allowing for people to filter out conversation they don’t find relevant. Explore is unique. It became clear to us that to grow, we need to create something designed for our community. Your responses to the survey were a great help in understanding what you want!

Security & Accessibility

From the moment we started the discussion about what a self-hosted comment system would mean, security and accessibility were at the top of our team’s mind. Part of accomplishing these goals will be consulting with experts to audit the system. Security experts will look at our system and assess it for vulnerability. We will design the system with accessibility in mind from the start. During the beta test we will specifically ask for accessibility testers to reveal any issues there. Once the system is live, we will create guides to help people navigate the new comments.

Do you currently use a commenting account on explore.org?

It actually came as somewhat of a surprise to me that 40% of people who took the time to participate in the survey do not comment. The next question explains why people don’t comment. 

If you said no, why don’t you use a commenting account?

To analyze the results of this question, I removed everyone who answered, “I do comment.” We accidentally made this a required question. Sorry about that. I then pulled out all the “other” answers and found a few common themes. 

By far the biggest reason people don’t comment is simply that they don’t want to, or are content reading the comments. The rest of this result confirms our suspicion that the bugs in Disqus are preventing people from being able to comment. This validates our goal of trying to build a stable commenting system first. The rest of the “other” responses have given us a lot to think about. For those of you who cited security concerns, we fully agree that the security of our users’ information is vital. I can confidently say that explore does not, and will never, sell our users’ information. It also looks like we could be better at educating our community about what is possible on the site. I was personally surprised that there were a number of people who didn’t know how to comment or didn’t know they even could comment on explore. 

If you use a commenting account, what is your favorite feature?

To analyze this question, I did something similar to the previous question. I removed all the people who said that they don’t comment. I also broke out everyone who answered “other” and organized the answers around the themes that emerged. 

We learned a couple things from this question. First, we confirmed our hypothesis that explore.org would not be a good fit for a scrolling chat like Youtube, twitch, and other live streaming platforms. The ability to leave and come back is essential to our community. We also learned that while many of you love the ability to post snapshots to comments, it has been deeply frustrating that that feature has been unstable at best. We agree. The reason it hasn’t worked recently is that Disqus’ system flags most attempts to post a snapshot to comments from explore as spam. If we host our own system, we will obviously not flag our own traffic as spam. This would be a very easy win for explore staff, moderators and fans alike. 

If I were to change one thing about the commenting system, it would be

This was one of the more insightful questions in the survey! Like the previous questions, I sorted the “other” responses into themes. There were quite a few pointing out specific bugs in the Disqus. I sorted the bug complaints into the label, “a stable system that works.”

By far most people wanted an easier way to find information they are interested in. The responses in the Other section came with very helpful suggestions. We are currently discussing search and filter options that would let people more easily find what they are looking for! Many of these suggested features are also found in the free-answer portion of the survey, so I will address more of them there. 

There were a small minority of people who urged us not to move away from Disqus. While we are happy that you have not experienced issues, we hope that this blog helps you see the extensive problems most of the community face. We want to reassure people that our goal is to make the transition as simple and painless as possible. Ideally you will be able to comment on the new system with your existing explore.org account and won’t have to sign up for anything new. As much as possible, we aim to maintain any existing features that work, while improving their stability. Any additional features we add will be tested to make sure they add value to the community.

“How Interested Are You” Questions

These questions were meant to gauge the interest of the community on some features that the explore staff brainstormed in one of our planning meetings. While the primary goal is to create a more stable system, we have the opportunity here to add features that could be fun and useful. As a reminder, 1 is Very Disinterested and 5 is Very Interested.

This response was actually somewhat surprising to us! We thought that the ability to private message friends would be a good way to allow for off-topic chatter without making it visible for those that were disinterested, but 56.8% of people were either very disinterested or disinterested in this feature. Only 24.5% of people were interested or very interested in the feature. This was very helpful feedback. Private messaging adds a layer of complexity to the development process. With such a strong response against private messages it is unlikely we will release this feature in the first phase of the new comment system. There were many responses in the free answer portion that specifically asked for private messages. If you were one of those people, know that this doesn’t mean we won’t ever support private messages. It just means that it will not be prioritized over some of the other highly requested features like expanded search and comment filtering. 

This feature was suggested in response to a somewhat common complaint about the phenomena of multiple posts of the same photo in chat. We thought if people could curate multi-photo posts it might incentivise them to curate one large post instead of posting a lot of the same photo. The response here is split. 39.7% of people were some form of disinterested, 36.7% had some form of interest. With a split response, the release of this feature will come down to ease of development and implementation. We might ask for more feedback on multi-photo posts when we have a more concrete example to show people. 

To be honest, this is a feature that the explore staff is very strongly interested in. We use the Fan Favorites section to create the Fan Cam Friday newsletter. As it stands, it is not super obvious how to favorite other community members’ snapshots. I would not be surprised if some people learn right now from this blog post that they can go to a gallery and favorite other people’s snapshots. Moving this feature directly to the comments would add another way to interact with your friends AND it would help us curate the Friday newsletter better. With 36% of the people disinterested and 42% of people interested, we will probably move forward with this feature!

I actually expected this question to even more strongly favor the Very Interested side than it did. 36.5% of the people were disinterested while 44.1% were interested. We know that comment archiving is critical for several fan wikis. We are exploring ways to import Disqus history into the new comment system. We are currently discussing how long we should archive comments in the new system. Text is fairly inexpensive to store. Addition of media such as photos, videos, and gifs, adds to the eventual storage costs of a self-hosted system. We are trying to find a way to balance the need to archive comments with operational costs. Rest assured we are planning on archiving comments, though for how long is an open question. We might request more feedback later in the development process to understand how you are currently using comment archives. 

This feature was suggested as a way to use the snapshot galleries in a new fun way. 37.5% of the respondents were disinterested while 40% of the respondents were interested. Several people mentioned in the free-answer portion of this survey that this would be a great way to discuss bear IDs. It’s a fantastic observation that we didn’t consider! 

We often get feedback that the explore.org app would be better if it had comments. 32% of people were disinterested while 44% of the people were interested. We have tried to get Disqus to work with our app multiple times over the years, and it just has not worked. App integration is very high on our feature requirements for a new comment system. 

We suggested this feature because we know that many schools and parents have their kids watch explore. It is not a huge surprise that 69.4% respondents were not interested in this feature while only 13.4% were interested. Explore’s website audience does trend to people who no longer have school age children. This strong response against parental controls means that it will not be very high on the feature list. Again, this does not mean that we will never release this feature, only that it will not be prioritized in the beginning.

Is there anything else you’d like to see out of a new comment system?

Personally, this was the most interesting and insightful part of the whole survey. Some of the features suggested here we have already addressed. I won’t address every suggested feature, but I would like to highlight a few. 

Stability: 

This has been addressed throughout this blog. It is our primary goal for a new comment system. The ongoing bugs are detrimental to the community. They are also the primary source of complaints for the explore.org website. 

Search & Filtering

These suggestions were the best thing to come out of this survey. So many of you had great ideas for how to use search and filtering to make information easier to find, and to interact easier with other fans. We are currently brainstorming an “off-topic” tag that would allow people to filter out off-topic comments while creating space for people to interact with their friends. 

Off Topic Comments

I did say at the top of the blog that we were not addressing moderation policy issues, but I also acknowledged that the off-topic tension is a symptom of the limitations of Disqus. We obviously want people to feel like they are welcome on explore and we understand that many longtime friendships have been formed here. On the other hand, we also got a lot of feedback from people who say that the off-topic comments make them feel unwelcome, and distract them from the goal of learning more about the live cams. It is a difficult balance. We hope that some of the search and filtering options can create room for both perspectives. 

No Repeated Snapshots 

This was also a common complaint. We don’t have any solid solutions here yet but we are actively considering how to address this issue. In a similar vein, many people requested a way to hide media. We think a filter that would let you see Images Only or Text Only might be a good solution. 

Reactions & Emoji’s

This was also suggested in one of explore’s brainstorming meetings! It is something we are also interested in developing.

Block and Unblock users

This was on our list of basic feature requirements already. Not being able to unblock a user on Disqus is confusing to us as well.  

No Ads

This came up a few times in the survey. Explore does not, and will never, advertise on the site. If you are currently seeing ads in Disqus please let us know, because they should not be doing that. We can guarantee that there will not be ads in the new product. 

Conclusion

Thank you to everyone who made it this far! This will be a long process, but we thank you for helping us start this journey. I didn’t address every feature or concern raised in the survey here, but we have read all of them. Your feedback is critical in making a comment system everyone will enjoy. We will continue to provide updates as we move forward. As always, if you have any questions, comments or concerns, please email us at feedback@explore.org