1、The Productive Postdoc: Do Working Conditions Affect Outcomes?,Geoff Davis Visiting Scholar and Survey Principal Investigator Sigma Xi, The Scientific Research Society gdavissigmaxi.org,Improving the Postdoctoral Experience,Many calls for changes to the postdoc National Academies, AAU, NPA, etcBig q
2、uestion: What, if anything, works?,What Works?,Changes have costs (money, time) Do benefits justify investments? What should priorities be? What gives the biggest bang for the buck?These are empirical questions,Our “Experiment”,Postdoc administration takes place largely at the level of the PI Tremen
3、dous variability in conditions from lab to lab Recent, limited introduction of new practices Natural experiment Ask postdocs about their working conditions Ask about how well they are doing Find conditions associated with positive outcomes,Sigma Xi Postdoc Survey,Ran a big web survey Contacted 22,40
4、0 postdocs at 47 institutions 40% of all postdocs in USOverall response rate: 38%* (*See tech report for details),Our Sponsor,The Alfred P. Sloan Foundation,Alfred P. Sloan,Michael Teitelbaum,Additional Support,Werthheim Fellowship, Harvard University,Partner Organizations,National Postdoc Associati
5、onSciences Next WaveNBER/Sloan Scientific Workforce Group,Sketch of Our Analysis,Create measures of inputs (working conditions, demographics, etc) and outcomesBuild linear models to test hypothesis that inputs have an impact, gauge magnitude of impact (if any),How Do We Determine Success?,Ideal: tra
6、ck people down in 10 years, see what they are doing / have done Problems: Very expensive Takes 10 years to learn anything Driving via the rear view mirrorInstead, look at immediate proxies for longitudinal data,Outcomes,What makes for a “good” experience?No single “best” measure Different people wan
7、t different thingsCreate collection of outcome measures Look at impact of inputs on each,Subjective Outcome Measures,Subjective success measure Overall satisfaction, preparation for independent research, quality of training in research / teaching / managementAdvisor relations measure How is your adv
8、isor doing? Is s/he a mentor? How would s/he say you are doing?Generate numerical scores by summing Likert scored answers,Objective Outcome Measures,Absence of Conflict/Misconduct Has postdoc had a conflict with advisor? Has s/he seen misconduct in the lab?Productivity Rate at which papers submitted
9、 to peer reviewed journals,Outcome Measure Distributions,Outcome Measure Details,Correlations all fairly low Subjective success and advisor relations 0.45 Other pairwise correlations all 0.2,Our Explanatory Variables,Model outcomes as function of explanatory variables Field of research Institution B
10、asic demographic variables Sex Citizenship Minority/Majority Status Type of degree (MD vs PhD) Total time as a postdoc “Working Conditions”,“Working Conditions”,How do we measure working conditions? Inspiration comes from various calls for changes Look at rate of implementation,Recommended Changes,5
11、 broad classes of recommended changes Pay people more Fellowships rather than assistantships Better benefits More structured oversight Transferable skills training,Measures of Working Conditions,Salary measure log(annual salary), full-time people only Independent Funding measure Dummy variable, 1 if
12、 fellowship, 0 otherwise Benefits measure Count of different benefits received (health insurance, retirement plan, etc),Structured Oversight,Structured Oversight measure Count of administrative measures in place Individual development plans Formal reviews Policies (authorship / misconduct / IP / etc
13、) Letters of appointment High values = lots of structure, low = little,Training,Transferable Skills Training measure Count of areas in which postdoc reports receiving training Grant writing, project/lab management, exposure to non-academic careers, negotiation, conflict resolution, English language,
14、 etc High values = training in lots of areas Low values = no training in lots of areas,Working Conditions Distributions,Working Conditions Details,Again, correlations all fairly low Structured oversight and skills training 0.30 Other pairwise correlations all 0.15,What Has Biggest Impact?,Who is mos
15、t satisfied, most productive, etc? People with Independent funding? High salaries? Lots of benefits? Lots of structured oversight? Lots of types of transferable training?,Simple Analysis,Crude analysis: compare satisfaction, productivity, etc for people in appointments with Fellowships / other fundi
16、ng High / low salaries High / low benefits High / low structure High / low training,Independent Funding,Salary,Benefits,Structured Oversight,Transferable Skills Training,Regression Coefficients,Take Home Message #1,Structured oversight and transferable skills training make a big difference,Causality
17、?,We have correlation. Is there causation? Psych literature gives reasons to believe in causationAlternative explanations Structure and training attract people who are intrinsically more satisfied / productive / successful Structure / training correlate with some other unobserved factor Advisors are
18、 effective managers / have more resources Postdocs take more initiative / are better organized / etc,Causality?,2 classes of explanation Structure/training attract intrinsically more productive people Structure/training directly cause productivity or are indicators for some causal mechanism (Some co
19、mbination of 1 & 2 also possible) Should be able to differentiate between 1 & 2 by looking at people with multiple appointments,Intrinsic vs. Time-Localized,Causality?,Add in terms that allow for change in slope of papers(t) curve starting at beginning of most recent postdoc Equivalent to adding int
20、eractions with ratio (months in current postdoc / total months as postdoc) to regression modelTraining appears to have a time-localized effect Other inputs ambiguous,Dont Pay Postdocs?,Not saying postdocs shouldnt be paid! Hard to attract US students to science if you dont pay them Maslows hierarchy
21、 of needs Must meet basic physical security needs first Living wage, basic benefits More nuanced interpretation of data: beyond a certain threshold, structure and training matter more than compensationInstitutional “postdoc tax” to support service provision?,More Details,Look at individual component
22、s of structure and training measure What specific measures have the greatest impact?,Impact,One measure appears to have significant impact all 4 outcomes: Research / career plans Written plans Plans that spell out what both postdoc and PI will doAdvocated by FASEB, National Academies,Plans,Compare t
23、hose with such a plan to those without: Much less likely (40%) to be dissatisfied Much less likely (30%) to have conflicts After controlling for field, institution, demographics: Submitted 14% more papers for publication,Why?,Plans: Expectation setting device Postdocs without plans were much more li
24、kely to report PI had not lived up to expectations Contract Research shows that people are more likely to live up to explicit (esp. written) commitments Forces postdocs to take responsibility for their careers early More time to take advantage of training opportunities Time management device Mechani
25、sm for focusing effort,Take Home Message #2,Individual development plans make a big difference,Additional Measures,Several other measures show concrete benefits: Teaching experience Exposure to non-academic careers Training in proposal writing Training in project management Training in ethics,Policy
26、 Implications,For postdocs, more effective to invest additional dollars in management than in salaries Management at all levels: Infrastructure for institutional oversight / training Management training for PIs Management training for postdocs,Further information,More information athttp:/postdoc.sig
27、maxi.orgWorkshop (with NPA) in January 2006 Contacts Geoff Davis, PI, gdavissigmaxi.org Jenny Zilaro, Project Manager, jzilarosigmaxi.org,Extra Material,End Products,Sigma Xi: Highlights in May/June issue of American Scientist Tech reports (2 out now, more to come) Scholarly paper this fall NPA: Ana
28、lyses of various topics NBER SEWPWorkshop in January 2006,Aside: Postdoc Definition,Half a dozen different definitions AAMC, AAU, FASEB, NAS, NSF BUT if you read and compare them, they all say the same thing Only substantive difference is that FASEB includes narrow subset of clinical fellows (We exc
29、luded them from this analysis)Most people dont fully satisfy definition anyway,Postdoc Definition,The appointee has a PhD or equivalent degree, the degree was received recently, the appointment is temporary, the purpose of the appointment is training for a research career, the appointment involves s
30、ubstantially full-time research or scholarship, the appointee is expected to publish the results of his or her research, and the appointee works under the supervision of a senior scholar or a department in a university or research institution.,Survey Non-Response,30-second summary of non-response an
31、alysis: Non-citizens and African Americans appear to be slightly under-represented No evidence of bias based on level of satisfaction (respondents not overly disgruntled),Survey Non-Response,Survey respondents atypical in one important way Participating institutions all had PDO, PDA, or administrato
32、r interested in postdoc affairs Participating institutions probably better off than average,Salaries,Median salary: $38,000Up from $28,000 in 1995,Inflation,A 10% increase above inflation since 1995 ($28,000 in 1995 = $34,700 in 2004)NIH budget doubled over the same period (in inflation-adjusted dol
33、lars),Experience,Salaries increase at about 2.9% per year of experience,Field,Overall average = $39,300Average salary in most common fields ranges from $37,500 to $40,000 Higher: Electrical engineering ($45,000) Physics ($42,600) Oncology ($41,400) Materials science ($41,200) Lower: Ecology ($35,600
34、),Institution Type,Govt labs pay 20% more than average Public universities pay 9% less than average,Taxes,Tax loophole: some postdocs dont have to pay FICA (7.65% of income) 23% benefit New IRS rules affect this Tax penalty: some postdocs pay extra self-employment tax (also 7.65% of income) 12% pay
35、Independent contractor status carries hidden tax penalty!Potential $6,000 impact on salary,Part-time,3% report part-time statusAverage hours worked previous week: 45,Hours,51 hours/week medianPostdoc hourly wage $14.90,Hours,51 hours/week medianPostdoc hourly wages = $14.90/hourHarvard janitors = $1
36、4.00/hour,Foreign Postdocs,International Men and Women of Mystery,Basic Demographics,Citizenship: Citizens: 40% Permanent residents: 6% Temporary visa holders: 54% PhD: US PhD: 53% Non-US: 47%,Non-US PhDs,Where PhD earned:Almost 80% of postdocs on temporary visas earned their PhDs outside the US Non
37、-US PhDs invisible in NSF stats,Non-US PhDs,Where non-US PhDs were earned: Country of citizenship 86% Different country, same continent 7% Different continent 7%,Temporary Visa Holders,Non-US Postdocs and PhDs,China and India dominate Market share of postdocs comparable to share of doctorates (China
38、 = 23%, India = 10%) Next largest LDC is Argentina, #16 for both citizenship and PhDs, with 1% of each,Temporary Visa Holders by Field,Broad Field,Other Characteristics,US postdocs: 49% men/51% women69% married 33% have children Median age: 33,International postdocs: 65% men/35% women69% married 35%
39、 have children Median age: 33,Other Characteristics,One notable difference for married postdocs US postdocs: 15% have non-working spouse Non-citizen postdocs: 44% have non-working spouse Some visas (e.g. H) dont have provision for spouse to work,Domestic vs International: Papers,International postdo
40、cs publish more Average peer-reviewed publications as a postdoc Citizens/PR 2.6 Temporary 3.3 (27% more)Difference is smaller (.1 papers/year) after we control for time as a postdoc, field, institution, sex, but statistically significant,Domestic vs International: Hours,Non-citizens work longer hour
41、s Average weekly hours worked Citizens/PR 50 Temporary 52 (4% more)Difference is smaller (1.3 hours/week) after we control for time as a postdoc, field, institution, sex, but still statistically significant,Domestic vs International: Salary,BUT non-citizens are paid substantially less Median annual
42、salary Domestic $40,000 International $37,000 (8% less)Domestic postdocs earn $2,200/year more than international postdocs after controlling for field, institution, sex, time as a postdoc, and funding mechanism,Domestic vs International: Grants,Citizens write more grant proposals (results suggest mo
43、stly fellowship applications) Grant proposals written while a postdoc Citizens 1.6 Non-citizens 1.1 (31% fewer)International postdocs write fewer grant proposals even after controlling for field, institution, sex,Domestic vs International: Satisfaction,Non-citizens report slightly lower levels of sa
44、tisfaction with the postdoc experience Average satisfaction (-2 = dissatisfied / 2 = satisfied) Citizens/PR 0.8 Temporary 0.6Difference disappears when one controls for salary, discipline, institution, sex, and time as a postdoc,Security Problems,To what extent have US national security regulations
45、affected your ability to do the following: (% responding “Some” or “A lot”) Conduct your research in the US: 30% Travel outside the US to conduct your research: 40% Visit your country of citizenship: 55% Re-enter the US after leaving the country: 57% Bring your immediate family members to the US: 36
46、% Free-text comments express considerable frustration,More information,More information athttp:/postdoc.sigmaxi.orgContacts Geoff Davis, PI, gdavissigmaxi.org Jenny Zilaro, Project Manager, jzilarosigmaxi.org,Survey Responders,Difficult to obtain ground truth for assessing results Plan: compare resu
47、lts of pilot survey to known values for one institution with good records Reality: survey revealed that the institution in question was missing lots of postdocs (10% of the local population),Survey Responders,Fortunately we found an alternative with better records Differences in response rates consi
48、stent with levels of variation in a random sample for Sex Citizenship Minority status No strong evidence of non-response bias,Further Non-response Analysis,Survey literature: propensity to respond is a continuous variable Early responders: high propensity Late responders: lower Non-responders: lowes
49、t Idea is that non-responders are more similar to late responders than early responders Compare early and late responders. Differences suggest potential non-response bias.,Non-response Bias?,Who are missing 66% of postdocs? No significant difference between early and late responders by Sex Overall s
50、atisfaction Significant but small difference by citizenship (p 0.04) Early responders: 49% citizens Late responders: 45% citizensNon-citizen postdocs are probably slightly underrepresented,Domestic vs International: Satisfaction,Non-citizens report slightly lower levels of satisfaction with the postdoc experience Average satisfaction (-2 = dissatisfied / 2 = satisfied) Citizens/PR 0.8 Temporary 0.6Difference disappears when one controls for salary, discipline, institution, sex, and time as a postdoc,
