research methods, 56:834:535:01 [current syllabus at https://theaok.github.io/res]
Fall 2024; Tue 6:00-8:50pm, ATG-201

  • Adam Okulicz-Kozaryn adam.okulicz.kozaryn@gmail.com
  • office: office hours: Tue, Thu 1-2, and by appointment: 321 cooper st 1st fl lab in the back
    • assistant:
  • Wei Chen wc642@camden.rutgers.edu
  • office: 321 Cooper St, Room 202 (2nd Floor); office hours: Thu 2-3, and by appointment

    • mission/philosophy

    Do talk to me about challenges at the beginning! I can accommodate much of it, if you tell me early!

    I will treat research methods/skills extremely broadly in this class: you can do research in great variety of ways!! Everyone is different (especially in this class)! Many ways to excel (eg opportunities for extra credit): some will excel by doing math, statistical programming, doing civic engagement, having great research ideas, if you are motivated you will excel one way or the other! if you fail in one area, you can make it up in another area!

    course description

    Data are everywhere: in public and private sectors, even in academia. People and organizations need to be able to make sense of data and understand others making sense of data.

    This is a graduate introductory quantitative research methods class. It will focus on understanding or consumption (less production) of applied quantitative analysis for social science. The goal is that you become a reasonably informed consumer of basic quantitative information, and also that you will be able to generate basic quantitative information.

    Are you more of a researcher, data person, analytical mind? Then produce research in this class, eg use Excel, R, or Python.
    Or are you more of a manager or consumer of research? then do literature review and critique others' research (which needs to do either way first)

    Some research design and research process will be discussed as well.

    We will go as slow as necessary, and try to have plenty of in-class discussion, presentations, and will flip parts of some classes.

    course objectives

  • review and critique academic research (done by others)
  • calculate basic statistics such as mean, median, and mode
  • understand research process
  • understand quantitative information; eg basic concepts of research design, descriptive and inferential statistics
  • understand graphs

    • required books

    Wheelan Naked Statistics: Stripping the Dread from the Data 1st Edition by Charles Wheelan; note: when i assign chapters, do read the appendices

    Trochim http://www.socialresearchmethods.net/kb/contents.php

    bonus books for advanced users/math aficionados

    OpenIntro http://www.openintro.org/stat/textbook.php 4th ed

    software

    Unfortunately, none. BUT, you are encouraged to see me outside of the class, where we can have a look at excel, stata, python, r, etc. It is 21st century, and to produce research, you really need to use a computer.

    grading

  • problem sets 60% (6ps*10points) (some of ps will be in-class presentation)
  • final presentation 30%
  • class participation/discussions [incl seeing me/emails say if you are shy] 10% [ updating the grade weekly or biweekly so you can track your progress and adjust]
    • min max grade
    90.0100.0A
    85.089.9B+
    80.084.9B
    75.079.9C+
    70.074.9C
    069.9F

    academic calendar

    tentative, most uptodate always on the class website: I work on those materials continuously and they will be changing slightly
    you may also want to see university calendar
    you want to print several slides on one sheet, say 6
    or just annotate electronic copy
    [*] means bonus (extra/not required)

    sep3 introduction to course and data: why research methods? [vid] [v old vid] [old vid]

    go over the syllabus, briefly discuss basics of research methods
  • ps1.pdf
  • intro.pdf
  • res_des.pdf: intuition sec only [we'll do everything in 2nd half of semester]

    • data basics and descriptive statistics

    sep10 descriptive statistics 1: terminology and theory [vid] [v old vid] [old vid]

    we will dive into descriptive statistics and continue next week
  • ps2.pdf
  • des1.pdf
  • https://www.nytimes.com/interactive/2017/12/08/us/puerto-rico-hurricane-maria-death-toll.html: measurement is the key! GIGO! it is critical how you measure it; ideally do it in many ways, ie triangulate
  • Wheelan: Introduction, ch1-3
  • Trochim: Conceptualizing http://www.socialresearchmethods.net/kb/resprob.php
  • Trochim: Unit of Analysis http://www.socialresearchmethods.net/kb/unitanal.php
  • Trochim: Variables http://www.socialresearchmethods.net/kb/variable.php
  • Trochim: Descriptive Statistics http://www.socialresearchmethods.net/kb/statdesc.php

    • sep17 descriptive statistics 1-1: relationships in data [vid] [old vid]

  • DataBasics.pdf
  • des1-1.pdf (also spend like 30min on ps1, brainstorm, discuss)
  • hows ps2 going? doing calculations in excel or what?
  • Wheelan: ch4
  • Trochim: Types of Relationships http://www.socialresearchmethods.net/kb/relation.php
  • [*] Trochim: Correlation http://www.socialresearchmethods.net/kb/statcorr.php

    • sep24 descriptive statistics 2: practicing, mostly graphs vid [v old vid] [old vid]

  • des2.pdf
  • quick discussion of ps2--anyone would like to present?
  • revisit anything, say hist/distr, crosstabs, scatterplots https://www.mathsisfun.com/data/correlation.html, q and a
  • btw, totally fine to change topic as you keep going, it evolves (but change sooner than later; and less so later (more difficult and costly for you)); great if you are finding sth unexpected

    • oct1 ps2 presentations 5min sharp, no more than 5, max 7 slides/displays; 5min discussion vid

    research design

    oct8 research design and hypothesis testing vid [v old vid] [old vid]

  • ps3.pdf
  • go over general ps2 comments appended to ps2.pdf (specific comments in your canvas)
  • find research: overview and practice of google scholar http://theaok.github.io/generic/howToGoogSch.html [~1hr] and lets do some queries of your research
  • res_des.pdf
  • trochim: foundations and design
  • Wheelan ch7, 8
  • [*] Wheelan ch9,10
  • [*] trochim: measurement; sampling
  • [*] trochim: Introduction to Evaluation http://www.socialresearchmethods.net/kb/intreval.php
  • [*] a fantastic webbook for research design (and methods, in general): i encourage you to browse through it and read more than just chapters specifically referred to here http://www.socialresearchmethods.net

    • oct15 more about research design and hypothesis testing [old vid] vid

  • ps4.pdf [remember about ps3 ]
  • revisit Google Scholar
  • continue with last weeks slides pick up with 'threats to internal validity'
  • flip the class: work on ps4, and possibly ps3
  • final_project.pdf [at least skim through TOC]
  • Wheelan ch13, Conclusion

    • oct22 examples of research [vid] [v old vid] [old vid]

  • pick up with around slide 43 "discontinuity analysis (p.238 Wheelan, 2013)"
  • discuss Wheelan from last 2 weeks
  • rand_slides.pdf : QOL in SJ: go quickly, skip bar charts showing 2000-10 chng; also drug abuse in NJ: http://www.nj.gov/humanservices/dmhas/publications/statistical/Substance%20Abuse%20Overview/2015/statewide.pdf note: p4 percents are column percents (sum up to 100 within columns); a quick critique: just admission counts, but different county pop, and many addicts not admitted
  • ex of succesful paper (papers used to be required, now presentation only; still presentation will contain similar key outputs, eg graphs, tables (note: these ex took production route: produced stats; again it is ok to just critique research) paperExample1.pdf; paperExample2.pdf
  • 2024 capstones nominated for award capstone_1.pdf; capstone_2.pdf; capstone_3.pdf
  • if time: flip he class and work on ps3,4

    • oct29 presentations: ps3/ps4 5min per person (2-person group 7min) sharp! (i will cut you off) vid [old vid]

    nov5 intro to regression [vid] [v old vid] [old vid]

  • ps5.pdf
  • reg1.pdf, and will continue next week
  • continue with presentations: Tyler, Kiara, Tammy, Rich
  • [*] https://colab.research.google.com/github/theaok/vis/blob/main/resMet.ipynb
  • Wheelan ch11

    • nov12 continue with regression [vid] [old vid]

  • revisit Python notebook, and pick up with slide "predicted values (p200 Wheelan, 2013)"
  • City Life: Rankings (Livability) Versus Perceptions (Satisfaction): mostly fig, tab, essp rankings and corr
  • More Unequal In Income, More Unequal in Wellbeing: fig1, tab1; tab2 and see Wheelan p33
  • alberto alesina: the more diversity, the less spending: see regressions in "PUBLIC GOODS AND ETHNIC DIVISIONS"; may also see (though not OLS but SUR): https://www.nber.org/system/files/working_papers/w10313/w10313.pdf and https://www.nber.org/system/files/working_papers/w26620/w26620.pdf and https://link.springer.com/content/pdf/10.1023/A:1024471506938.pdf
  • I may post few more; let me know if you know of any good ones!
  • may reread Wheelan ch11
  • Wheelan ch12
  • flip the class work on ps5; anyone show what you have so far?; troubles frinding reg tables?

    • nov19 probability [vid]

  • ps6.pdf
  • prob.pdf
  • Wheelan ch5-ch6
  • try to flip the last 20min or so: discussion, questions, work together on ps

    • nov26: no class : Change in Designation of Class Day--Observe THURSDAY schedule

    dec3 final presentations: 11min (1 extra min v next wk) sharp! (and 15min discussion (3 extra min v next week)) vid

    [old vid] Make it legible! Make it as big as necessary for people to be able to see easily; dont go too fast, again max 1slide per minute

    Most of the final presentation must be about *your* research either what you did or plan to do, not what others did. Its about your research, not others!! Less about background and others, more about YOUR study; simplify and streamline! Do not put too many regressions by others in your final project!

    Your project does not have to be fancy Python with huge raw datasets; can be cross-tabs with data from published research, blogs, Wikipedia, etc. But you need to do something new, creative, innovative, and be rigorous; and have to use some concepts from the class; (again can be just lit rev, but then synthesis (not just summary), comprehensive and still do add value, eg extract best practices).

    see sec 'examples' for examples of a perfect slide for presentation (just one slide, not whole presentation)
  • discussion of last 2 ps on probability and regression (generic comments appended to ps pdf!); may revisit reg1.pdf and prob.pdf

    • dec10 continue presentations 10min sharp (+12min discussion) [old vid]

  • wrapUp.pdf if time: brief course summary
  • Wheelan ch12 and conclusion
  • [*] equality and equity: Brandi Blessett, Counternarratives as Critical Perspectives in Public Administration Curricula; Brandi Blessett, Social Equity in Public Administration: A Call to Action
    • ad: https://theaok.github.io/vis/

    rules

    24fa NASPAA competency: To analyze, synthesize, think critically, solve problems and make decisions

    do not share or link to class videos! These videocasts and podcasts are the exclusive copyrighted property of Rutgers University and the Professor teaching the course. Rutgers University and the Professor grant you a license only to replay them for your own personal use during the course. Sharing them with others (including other students), reproducing, distributing, or posting any part of them elsewhere -- including but not limited to any internet site -- will be treated as a copyright violation and an offense against the honesty provisions of the Code of Student Conduct. Furthermore, for Law Students, this will be reported by the Law School to the licensing authorities in any jurisdiction in which you may apply to the bar.

    doing research with humans and other animals If you use data collected by someone else, you are fine. If you collect your own data, do experiments, or in any way do anything with human subjects or other animals, google 'rutgers irb'

    confidential data If you have, handle, or plan to use or collect any data that is in any way confidential, eg: SSN, DOB, phone\#, address, name, etc: talk to me first!

    attendance Attendance is mandatory: there will be quizzes and if you miss them, you miss credit. You are responsible for everything covered, including lecture, discussions, announcements etc. Attendance is usually essential to learning. If you miss a class, consult with a fellow student to learn what transpired.

    incompletes: Like late problem sets, only allowed in documented emergencies (eg hospitalization).

    study groups. Many students over the years have found the study groups to be very helpful. Study groups are permitted and encouraged to work on the problem sets together. However, each individual student must write up his or her own answer to hand in, based on his or her own understanding of the material. Do not hand in a copy of another person's problem set, even a member of your own group. Writing up your own answer helps you to internalize the group discussions and is a crucial step in the learning process. Also, if worked in group, spell out group members' names next to your name. You can work on problem sets and on the presentation in a group of upto 3 people. So your group would hand in one presentation, but each group member would hand in her own problem set that must differ from others [except problem sets that are drafts of final project; essp the later ones].

    academic integrity. I am very serious about this. Make no mistake--I may appear accommodating and informal--but I am extremely strict about academic integrity. Violations of academic integrity include cheating on tests or handing in assignments that do not reflect your own work and/or the work of a study group in which you actively participated. Handing in your own work that was performed not for this class (eg other class, any other project) is cheating, too. I have a policy of zero tolerance for cheating. Violations will be referred to the appropriate university authorities. For more information see http://fas.camden.rutgers.edu/student-experience/academic-integrity-policy

    accommodating students with disabilities. Any student with a disability affecting performance in the class should contact the disability office ASAP: google 'rutgers camden disability office'

    civic engagement component (opportunity for extra credit!)

    Start early. Start thinking about how you want to engage civically today.

    Note: if you are from far away, say North Jersey, let me know as you will be at some disadvantage, eg won't be able to take advantage of labs. And let's plan how we can make sure you can succeed in this class!

    typical civic engagement

    Universities and social science should serve society. You are encouraged have to engage with local community.

    The idea is that you engage civically using research methods. There are several ways to do it. Ideally, you will partner with a local organization, obtain data from them, do some analysis, and present results to them. You may also use government data, say from census bureau, and present relevant information to locals. A local organization can be Rutgers research institute such as WRI, CURE, LEAP or any other organization such as school or soup kitchen or CamConnect. Rutgers Office of civic engagement may be able to help you contact them. The key idea is partnership: you will use tools from this class to produce output useful to local community. This is similar to taking a role of an apprentice at a local organization or serving as a consultant.

    Using real world data poses challenges, which is a part of exercise. Presenting your findings to stakeholders outside of a class is also challenging. At the same time, it is fairly easy to contribute locally by using simple tools learned in this class. For instance, simple comparison of means between two schools in Camden can be revealing and helpful locally.

    An obvious way would be to use data at your workplace or at a workplace of someone you know. However, you need to make sure that it serves society in some way. For instance, it would be straightforward if you work at a hospital or school or fire department; but it would be difficult if you work at Starbucks.

    For instance, to give just one example, in the past some students worked for local schools and they obtained some data from Camden City Board of Education, and they simply interpreted those data and drew some conclusions. The idea is to evaluate the data more broadly and interpret it in a helpful and creative way, ideally connect to the literature and arrive at some conclusions that will help with the school operation.

    Some specific ideas: hopeworks, salvation army, and PBCIP.org

    atypical civic engagement--CONTACT ME FIRST if you consider this!

    Successful completion of atypical civic engagement will take estimated at least double of the typical civic engagement time.

    You could try to engage at regional or State level-for instance, you may evaluate some policy in NJ as compared to NY, or produce descriptive statistics of a region that would be useful regionally (eg my South Jersey WRI paper http://dept.camden.rutgers.edu/rand-institute/files/changes-across-the-region.pdf Such type of engagement typically requires substantial research experience typically found at late stage of PhD program. There may also be some other atypical ways-let me know your ideas.