Sampling Methods and Bias with Surveys: Crash Course Statistics #10

CrashCourse
28 Mar 201811:45
EducationalLearning
32 Likes 10 Comments

TLDRThis video explains non-experimental research methods like surveys and censuses which are used when experiments are unethical or impractical. It discusses pitfalls of surveys like biased questions, non-response bias, and problematic sampling. Tips are provided for identifying fake polls. The US census is explored as an attempt to survey an entire population despite great cost and difficulty. Well-executed non-experimental methods have value for businesses, countries, and content creators, but caution is urged regarding questionable polls.

Takeaways
  • 😀 Surveys are a common non-experimental method to collect data through questionnaires
  • 📊 Well-designed surveys have neutral, non-biased questions and reach a random, representative sample
  • ⏱️ The US Census aims to survey the entire population every 10 years, providing a huge dataset for analysis
  • 😟 Biased questions and sampling can skew survey results to match the surveyor's agenda
  • 🤔 Voluntary response bias means people with extreme views are more likely to take surveys
  • 🎯 Good survey questions offer comprehensive answer choices without 'leading' respondents
  • 🙅‍♂️ Unethical experiments like forcing people to marry or smoke can't be conducted, so surveys help fill research gaps
  • 🔎 Cluster and stratified sampling improve representation for minority groups in surveys
  • ✅ Census data provides a benchmark to compare against other studies and minimize sampling error
  • ❗Watch out for fake or fishy polls - check who conducted it and how before believing the results
Q & A

  • What are some examples of experiments that would be unethical to conduct?

    -The video mentions experiments that would randomly assign some people to be married and force others to remain single. Other unethical experiments could involve exposing people to harmful substances or denying them access to beneficial treatments without consent.

  • What are two key factors that determine the quality of a survey?

    -The two main factors are: 1) The questions asked in the survey, which should be clear, neutral and cover all options. 2) Who the survey is given to - the sample should represent the target population and be selected randomly.

  • How can the wording of a question bias the results of a survey?

    -Leading questions steer respondents towards giving a particular answer. Questions can also be biased by not providing a full range of answer choices or pushing people to respond in a socially desirable way rather than truthfully.

  • What is non-response bias?

    -Non-response bias occurs when certain types of people are more likely to complete a survey than others. For example, retirees may be more likely to do a phone survey during work hours. This skews the results if responders differ systematically from non-responders.

  • What is the purpose of using stratified random sampling?

    -With stratified sampling, the population is divided into key subgroups and random sampling is done within each group. This ensures that minority groups are adequately represented in the overall sample.

  • What are some of the challenges faced in conducting a census?

    -Challenges include the massive effort and cost to try to count every single resident. People can be missed, especially hard-to-reach groups like the homeless. In the past, lack of computing power also delayed results.

  • How are polls on the Internet and social media likely to be biased?

    -Internet polls suffer from voluntary response bias - only those with very positive or very negative opinions tend to respond. So results tend not to represent the average experience.

  • What advice is offered for spotting 'fake' polls?

    -Tips include checking if the poll seems professionally conducted, looking at who carried it out and their reputation, examining the survey questions and sampling methods, and watching for anything suspicious.

  • What are some appropriate uses for survey data?

    -Well-designed surveys can provide valuable information for businesses, governments, research and more - especially on topics that would be difficult to study experimentally. The key is proper, unbiased methodology.

  • How can analysis of census data differ from standard statistical analyses?

    -Since a census surveys an entire population, inferences about whether relationships exist are not needed. Analysis can instead focus on assessing if observed differences are meaningful in a practical sense.

Outlines
00:00
📺 Introducing the topic of non-experimental methods

The paragraph introduces non-experimental methods that can be used when experiments are unethical or infeasible. It gives examples like studying the effects of marriage or smoking. Non-experimental methods rely on observation rather than manipulation.

05:00
📝 Discussing surveys and sampling techniques

The paragraph focuses on surveys, including the importance of unbiased questions and proper sampling. It discusses techniques like simple random sampling, stratified sampling, cluster sampling, and snowball sampling. Sources of bias like non-response bias and voluntary response bias are also covered.

10:01
🗳 Explaining censuses compared to sampling

The paragraph differentiates a census that surveys an entire population from sampling. It talks about the US census, including its history, challenges, purposes, and analysis approaches. It also mentions why censuses are valuable despite high costs compared to sampling.

Mindmap
Keywords
💡survey
A survey is a non-experimental data collection method where respondents are asked a series of questions. Surveys are used to gather information about people's behaviors, attitudes, preferences, etc. In the video, surveys are discussed as an alternative to experiments when random assignment is unethical or infeasible. Examples of surveys from the video include website user experience surveys, political polls, and health questionnaires.
💡bias
Bias refers to a systematic error in data collection and analysis that skews results in a particular direction. Biased survey questions are discussed in the video as a major threat to the validity of survey findings. Examples include leading questions, questions with limited answer choices, and questions that make assumptions or push respondents towards certain responses.
💡sampling
Sampling refers to the process of selecting a subset of a population to survey. The video discusses proper sampling techniques like simple random sampling and stratified random sampling to obtain survey responses that are representative of the target population. Improper sampling can lead to biased and ungeneralizable results.
💡non-response bias
Non-response bias occurs when certain subgroups are less likely to respond to a survey, skewing the sample. For example, retirees may be overrepresented in a daytime phone survey. This can introduce bias if non-respondents differ systematically from respondents.
💡voluntary response bias
Voluntary response bias happens when the people who choose to respond to a voluntary survey are different from the overall target population. For example, online reviews tend to attract people with very positive or negative opinions rather than typical experiences.
💡census
A census attempts to survey an entire population rather than sampling. Censuses like the U.S. Census aim to provide benchmark data and reveal insights into subgroups that sampling might miss. However, achieving full participation is challenging and costly.
💡cluster sampling
Cluster sampling selects groups or clusters that occur naturally in a population and surveys some clusters randomly. This is more efficient than simple random sampling of individuals across a large population. For example, sampling some schools in a region rather than individual students.
💡snowball sampling
Snowball sampling uses current survey participants to recruit additional hard-to-reach respondents from the target population through their social networks. This is useful for studying small or difficult-to-access groups.
💡weighting
Weighting survey responses means counting some responses more than others during analysis to correct for underrepresentation and potential biases related to non-response or sampling. However, weighting can introduce other errors if assumptions are wrong.
💡inference
Statistical inference involves using a sample to draw conclusions about a wider population. With census data encompassing the full population, analysis focuses more on assessing the magnitude of observed differences rather than inferring whether relationships exist.
Highlights

Sometimes experiments that would provide valuable information are unethical or impossible to conduct.

Surveys are a common non-experimental method to collect data. The quality depends heavily on the questions asked and who answers them.

Survey questions should be worded neutrally without leading people towards certain answers.

Biased survey questions, whether intentional or not, skew results and limit the credibility of claims made from that data.

Getting surveys to a random, representative sample of a population helps improve data quality and limit biases.

Underrepresentation and voluntary response biases are common survey issues that can distort results away from population values.

Weighting responses and stratified sampling are techniques used to help account for underrepresented groups in survey samples.

Cluster sampling selects groups instead of individuals to make widespread surveying more feasible despite limitations.

Snowball sampling uses current respondents to recruit hard-to-reach populations for studies.

A census attempts to survey an entire population which provides very accurate data but at great effort and expense.

Census data allows for direct analysis of population differences unlike sampled data requiring statistical inference.

Be wary of fake, biased, unprofessional polls which can strongly skew perceptions of reality.

Well conducted surveys provide valuable information to organizations without requiring experiments.

Crash Course Statistics has its own survey in the description for viewers to check out.

The link to the Crash Course viewer survey takes less time than the extensive Nerdfighteria one.

Transcripts

Browse More Related Video

Techniques for random sampling and avoiding bias | Study design | AP Statistics | Khan Academy

Elementary Statistics - Chapter 1 Introduction to Statistics Part 2

Psychological Research: Crash Course Psychology #2

Research Design: Defining your Population and Sampling Strategy | Scribbr 🎓

Data Collection: Method types and tools

Stratified Sampling Vs Cluster Sampling with Examples | Meaning and Comparison

Rate This

5.0 / 5 (0 votes)

Thanks for rating: