🕑 2021-09-09 🖊 Phil Niehus

I conducted a survey to find out how well people can distinguish between sunrises and sunsets - Here are the results

Introduction

For a personal AI project I'm working on, I needed to know how well people can distinguish between sunsets and sunrises. I always found pictures of the sky during sunsets and sunrises hard to distinguish, but I wasn't sure if that was true for everyone. I needed numbers, so I decided to conduct a survey. I originally only wanted to survey some of my friends and family members, but the survey results weren't really conclusive, I needed more data. After some research, I found r/SampleSize and r/takemysurvey. Know that I found a way to survey a large quantity of people, I set about making my new survey. The following are the results of the survey.

Click here to download colorblind-friendly versions of the graphics in this article

Demographic

Before I start going into details about the demographics of this survey, I want to thank everyone who participated in the survey. I found all respondents for my survey on reddit by posting in r/SampleSize and r/takemysurvey, which I highly recommend to anyone looking for participants for their surveys.

Location

My survey received 95 responses from people out of at least 19 different countries. 22 respondents didn't specify their location. England and Scotland are combined into UK as part of the data normalisation and the Virgin Islands are included in the count for the US due to limitations of the plotly map. Here is an overview:

map of respondents locations Overview map of respondents locations. Click here for an interactive version.

Age

Most of the respondents where in the age groups of 18-24 and 25-34, which somewhat aligns with the overall age distribution of reddit users (Statista).

Responses per age group Survey responses per age group

How often respondents witness the sun rise/set

I also asked my respondents how often they witness sunsets and sunrises to find out if people who witness sunsets and sunrises more often are more likely to identify them correctly. The respondents where much more likely to frequently see the sun set than to frequently see the sun rise.

The percentage of respondents grouped by the frequency at which they witness the sun rise/set The percentage of respondents grouped by the frequency at which they witness the sun rise/set

People aged 25-34 are slightly more likely to frequently see the sunrise than people 18-24. I assume that this is caused by reportedly shorter sleep durations with increasing age (U.S. Bureau of Labor Statistics), (Withings). I don't have enough data to interpret for other age groups.

Sunrise encounter freqeuncy by age How often people witness the sunrise grouped by age

The statistics on how often people witness sunsets doesn't seem to have an age dependent trend.

Sunset frequency by age How often people witness the sunset grouped by age

Where this survey is applicable

The results of this survey only apply to pictures of the sky during sunrises and sunsets without any context. I assume that additional context in the picture would drastically increase the rate at which pictures are identified correctly as it would provide additional clues like human activity and location.

What the survey looked like

The survey included 2 types of questions:

  • Questions with just one picture of the sky
  • Questions with 2 pictures with one depicting a sunset and the other one depicting a sunrise
A two-oicture question A question of the second type (top: sunset, bottom: sunrise). Attributions: Credits 1), 2)

The purpose of the second type of questions was to find out, if the limitation that two pictures could only contain one image of each type would increase the recognition rate, which it didn't.

Results

Overall results

The respondents of the survey were on average able to identify 35.35% of the images correctly. Very few people were able to reliably identify the two categories in the pictures. This, together with the very low average of correct results, could very well mean that the people who had a higher percentage of correct guesses could just have been lucky, as the likelihood of a correct guess would have been higher if the respondents had just decided on a category by virtue of a coin toss. The highest percentage of correct guesses achieved by a respondent was 83.33%, the lowest was 0%.

Distribution of avg percentages of correct guesses The distribution of the average percentage of correct guesses per respondent.

The rate at which people answered each question correctly was consistently below 50%, with only one question that reached 61.05% of correct answers. An accuracy of at least 60% would have been required to be able to say that having people decide categories would lead to more reliable results than a random decision.

Percentage of correct answers per question Percentage of correct answers per question

Taking a look at detailed statistics

The thing I was most interested in when looking at the survey results, apart from the overall result, was seeing if there would be qualities that would make certain respondents more likely to give correct answers. Unfortunately there weren't any correlations between the frequency at which people witness sunrises or sunsets and their likelyhood to give correct answers. The group of people who witnesses sunrises a few times per week even has the highest deviation in their answers.

The average percentage of correct answers grouped by the frequency at which respondents witness sunrises The average percentage of correct answers grouped by the frequency at which respondents witness sunrises

The group of people who witness sunrises everyday was slightly more likely to make the right choice, but I would say that the difference in results is insignificant, especially considering the sample size of 3. The same also applies in regards to the respondents frequency of sunset observations:

>
sunset_freq responses correct_mean correct_median correct_std
0 A few times per month 22 39.394547 41.669998 14.360959
1 A few times per week 43 32.945351 33.330002 18.089376
2 A few times per year 11 34.090908 25.000000 22.191761
3 Almost never 6 29.165001 25.000000 21.570170
4 Every Day 13 40.385384 41.669998 16.261583
The average percentage of correct answers grouped by frequency of sunset encounters

The same is also true for age, so I'm not going to include detailed statistics about that here. Lets take a look at the most interesting question instead.

Question 5

The bottom half of this questions image isn't really all that interesting, but the top part is, because it's the only question that was more likely to be answered right than wrong. Lets take a look at the image I used for that question, before we discuss it's results.

Question 5 Question 5 (sunrise at the top, sunset at the bottom). Attribution: Credits 3), 4)

I personally would have had a really hard time deciding which of those two pictures is the sunrise. I assume people correctly picked the first picture because it looked slightly colder and darker, but that's just my personal opinion. Let's take a look at the data for the top half of this image.

sunrise_freq correct incorrect correct_percentage
0 A few times per month 15 5.0 75.000
1 A few times per week 6 4.0 60.000
2 A few times per year 18 13.0 58.065
3 Almost never 16 15.0 51.613
4 Every Day 3 0.0 100.000
Answers for question 5 grouped by sunrise encounter frequency

What's interesting about this question is, that people respondents who witness sunrises more often where more likely to answer it correctly. Which might explain why I found it hard to classify, since I am in the category of "a few times per year". The results for this question grouped by sunset encounter frequency look similar, so I won't show them here.

A short comparison with my control group survey

Before I ran this survey, I ran a very similar control survey with my friends and family members. It had fewer and slightly different questions, some questions were used in both surveys. 8 of the 10 respondents are in the 25-34 age group and 7 of the respondents have an education in STEAM fields (Science, Technology, Engineering, Arts and Math). The respondents of that survey were on average able to give correct answers in 58.89% of cases. Let's take a look at the results for the questions that both surveys shared.

Question % of correct answers
(control)
% of correct answers
(this survey)
Q1 50.00% 30.53%
Q3 10.00% 40.00%
Q7_bottom 20.00% 16.84%
Q6_top 20.00% 29.47%
A comparison between this survey and my control survey

I think the different performances for the same questions can be explained through demographics and question order. All the respondents of my control survey are German and are therefore used to a different climate than most of the respondents of this survey. The questions in this survey were also shown in random order, while the questions in my control survey were shown in a fixed order.

A zip-file with more statistics can be downloaded here.

Possible explanations of the results

Since there weren't any clear connections between different demographics and result quality, I can only attempt to explain the results. I assume that the lack of context in the pictures made it harder to give a correct answer, because it removed the important context of location. Different locations can influence the appearance of sunsets and sunrises due to different climates and different amounts of particles in the air. A higher particle count will lead to a more red appearance, since blue light is dispersed easier than red light and will therefore be perceived less intense in regions with higher counts of particles in the air.
The lack of context also removes the ability to make a guess of the current time of day on the basis of the amount of human activity. A qualitative comparison of a survey with pictures with context would be needed to confirm this theory.
Aligning with the science, people were more likely to classify pictures with a higher presence of reds and deep oranges as sunsets.

Conclusion

Humans aren't able to reliably distinguish between sunrises and sunsets from pictures of the sky without any additional context. The average rate of correct decisions of 35.35% means that random decisions would achieve similar or very likely better results. When a picture with a red sunrise is put next to a picture with a not very red sunset, people will select the wrong class for the pictures, which aligns with the science behind the color of sunsets and sunrises.

What the results mean for my AI project

My AI revolves around the assignment of multiple labels to pictures of the sky including labels for sunsets and sunrises. The purpose of this labeling is the selection of pictures of the sky from a large library of sky pictures that fit the look and mood of a given image for replacement of the sky in the image. Given humans' bad ability to distinguish between sunsets and sunrises without context, my AI model will not have to be able to distinguish between sunsets and sunrises. A single label for both will be enough.

  1. "Paris Sunset" by Mustang Joe is marked under CC0 1.0. To view the terms, visit https://creativecommons.org/licenses/cc0/1.0/
  2. "Sunrise in the upper Soda Butte valley" by YellowstoneNPS is marked under CC PDM 1.0. To view the terms, visit https://creativecommons.org/publicdomain/mark/1.0/
  3. "March 04, 2020 sunrise 7:17AM" by Katsujiro Maekawa is marked under CC0 1.0. To view the terms, visit https://creativecommons.org/licenses/cc0/1.0/
  4. "Sunset Sky" by Joshua Tree National Park is marked under CC PDM 1.0. To view the terms, visit https://creativecommons.org/publicdomain/mark/1.0/
  5. The members of r/SampleSize andr/takemysurvey who were great survey participants