So I guess this officially marks the start of the "unskewed polls" portion of this election.

this thread is fucking retarded

FiveThirtyEight is read by millions including thousands with advanced degrees in statistics, but you're the only one to catch his bias?

Yes? Someone has to be the first one to notice something. I have a PhD in political science, have taken quantitative methodology at the graduate level, and my dissertation did a lot with nonparametric inferential statistics. Nate Silver has a bachelor's degree.

And why does RealClearPolitics show an almost identical bump for Clinton?

They don't adjust for house effect one way or the other. There's no adjustment to manipulate. At 538 they turned corrections for pro-Clinton bias into corrections for pro-Trump bias, and they took existing corrections for pro-Trump bias and amplified them further. This was an across the board 8 point shift to the house effects of every pollster.

Let's say that you are correct, and 538 has been juggling the numbers to favor Clinton. Have you contacted any one there to discuss your findings? Are you going over this with any other statisticians to work out just what is going on?

Not yet, I just found this last night at 3am.

/u/setecordias might actually have a point, but sadly I think he has little understanding of how this shit actually works lol. You, OP (great job btw, im jealous, this reminds of the movie The Big Short) can contact all the local media, government, or authority figures you want, but the chances are slim that they will listen... Which is no reason to not try. Just trying to input some realism on behalf of the commentor

You missed the point of my question. It's one thing to make a case like OP made, gather all the evidence, and present it to an echo chamber where any one is unlikely to have any expertise to evaluate it. It's another, however, to present it to people who would have expertise in the matter and who could evaluate it. If OP has a PhD in statistics, then OP likely has a number of peers in the field who could honestly evaluate his evidence. If 538 made an error or committed some sort of fraud, then it is OP's ethical duty to present his analysis so they be given the opportunity to:

A. Become aware of any mistakes B. Correct any mistakes they made C. Correct any mistakes OP made D. Make an excuse of some sort

As Feynman said, "The first principle is that you must not fool yourself – and you are the easiest person to fool."

I sent an email to Nate earlier today from my university email, but no response so far.

I can show this to colleagues, but one of the difficulties of performing a "peer review" of sorts on 538 is that their model/formulas are proprietary and secret.

http://fivethirtyeight.com/features/the-conversation-about-skewed-polls-is-back/?ex_cid=2016-forecast

Thanks for this—apropos. They talk about criticisms of the polls themselves, and they mention that they "unskew" polls themselves by accounting for the house effect of each pollster. My concern is not about the polls themselves being skewed, but the 8-point shift in the way that 538 has been "unskewing" the house effect of the pollsters.

There's nothing nefarious here. 538 is a dynamic model. You're confusing the house effect with the trend line adjustment and other factors, which interpret data points in light of other data points available at the time the election simulations are run. As new data points become available (e.g., here, new polls showing Clinton ahead), this can change the significance of previous data points. You can argue with whether this kind of adjustment is methodologically sound (prior versions of the 538 model that used a version of this technique predicted 49/50 and 50/50 states correctly in 2008 and 2012), but bad methodology is quite different than intentionally skewing the numbers.

Re-read the section of the original methodology post concerning trend line adjustment and LOESS regression.

Also see this post by Nate Silver:

http://fivethirtyeight.com/features/election-update-why-our-model-is-bullish-on-trump-for-now/

On August 7th, the trendline adjustment was a total 2.7 points toward Clinton. Back on July 29th, the trendline adjustment had a negligible impact, as both candidates had roughly the same trend/slope. There certainly hasn't been a 7.9 point swing just in trendline adjustment in the past week.

Fucking bullshit, you obviously don't know the first thing about statistics. If 538 rigged the polls, why would they post all their 'rigging' for everyone to see? Do you think they want conspiratards like you bitching to them all the time?

Let me lay this out for you in simple terms. After the Republican convention, Trump was at +1, and currently he sits at about -7 according to RCP.

So the important part is that this is at the STATE LEVEL. Obviously adjustments at the national level would be bias. But if, at a time when national polls had Trump +1, Ohio polls had Trump +3, it stands to reason that when national polls have Clinton +7, The same Ohio poll should be interpreted as Clinton +5 because of the change in the national polls.

This is the "trend line adjustment" that Nate Silver so helpfully pointed out

Its sad that lying pathetic basement dwellers like yourself cook up these conspiracy theories to detract from actual questions people have about government and authority in this country. You even feel the need to propagandize your position ("Nate Pyrite" "I have a Ph D so listen to my inane rants"), truly you are the kind of scum people should be rallying against, not professionals who actually know how to do their job. If you want to "unskew" all the polls just ask Romney how that went.

What's even worse is your own self-righteousness. Not only did you have the kind of selfish stupidity that leads you to the conclusion that, despite 538's transparency and openness, you "discovered" some conspiracy and even tried to contact them, but you honestly believed you were the only person that noticed. You special fucking snowflake, you.

This is where I'll link you, because you're the kind of stupid outlined in this post. Not only are your claims easily disprovable by anyone, but you look so stupid as to detract from anyone with a legitimate gripe. God help us all when people like you exist.

The trendline adjustment for August 7th is listed at the bottom of the national poll page. It's +1.3 for Clinton, -1.4 for Trump. That's a total of 2.7 in favor of Clinton. The trendline difference on July 29th was negligible, because they both had the same very slight upward slope to their polling averages. Between July 29th and now, there hasn't been a 7.9 point swing in the trendline adjustment that would explain the differences I've highlighted.

Your username is decently fitting, as this must have taken a decent amount of work. Good on you.

General reminder that you (yes, you, reading this) need to take up responsibility in your local area to demand that your city government USE PAPER BALLOTS IN THE ELECTION. Voting machines must be terminated. Gather all the information required to convince the council of this and present it to them in a town hall or other public venue where others can hear.

[deleted]

Stop acting like a moron. Obama signs his death warrant if he suspends anything.

That will be the entire problem.

Why are you still talking about old news?

Why are you still shitposting?

[removed]

So no substance for your claims. Thanks for admitting you were just shitposting.

Nope.

Again, thanks for confirming that you are talking about the lack of substance.

Rule 10. Removed. Final warning.

Pollster bias isn't the only adjustment made in their model. It includes a likely voter screen, omitted third parties, trend lines, and allocation of undecided voters. State polls also include a demographic regression, and the polls-plus model includes economic data. The big swings you are seeing are largely from the trendline adjustments. Remember, 538 is trying to predict what the result will be Nov. 8, not today.

Lucky for you, they publish what those adjustments are:

National polls

Pennsylvania (as a state poll example)

On July 29th, the DNC had finished but few (if any) post-DNC polls had been released. The trend, at the time, was heavily in Trump's favor: the full effect of the email investigation and hearings brought Clinton down, and Trump had his own convention bounce and had (somewhat) solidified support from the Republican base. So, the trendline pointed to the race going in Trump's favor by election day, and the polls were adjusted towards Trump. Now, after the DNC convention, the trend show Clinton's lead growing rapidly, so the polls are adjusted in Clinton's favor instead.

There's nothing nefarious here. You haven't actually discovered anything new. 538 is a dynamic model. It's not retroactively adjusting numbers to fit a preferred narrative. Rather, the model always interprets data points in light of other data points available at the time the election simulations are run. As new data points become available (e.g.,new polls showing Clinton ahead), this can change the significance of previous data points. You can argue with whether this kind of adjustment is methodologically sound (prior versions of the 538 model that used a version of this technique predicted 49/50 and 50/50 states correctly in 2008 and 2012), but bad methodology is quite different than intentionally skewing the numbers.

Nate Silver explicitly stated that the model was doing this kind of adjustment all along. In fact, he described the adjustment in a post justifying why 538's model was more bullish on Trump than other models at the time:

Another tricky question is how to reconcile state polls with national polls. For example, there have been no polls of Pennsylvania over the past two weeks, during which time Clinton’s lead has evaporated in national polls (and often also in polls of other states, where we’ve gotten them). The FiveThirtyEight model uses what we call a trend-line adjustment to adjust those those old polls to catch up to the current trend. That’s why our polls-only forecast shows Pennsylvania as a tossup even though Trump has only led one poll there all year. Those older polls came from a time when Clinton led by 5 or 6 or 7 percentage points nationally, and they generally showed her up by about the same margin in Pennsylvania. Now that the national race is almost tied, it’s probably safe to assume that Pennsylvania is very close also. Some of the competing models don’t do this, and we think that’s probably a mistake, since it means their state-by-state forecasts will lag a few weeks behind, even when it’s obvious there’s been a big shift in the race.

Previously, the trend line adjustment favored Trump. Now it favors Clinton. It's driven by data and math, not pro-Clinton bias.

In the quote you provided Nate is discussing extrapolation of state polls using more recent national polls. This post is about the national polling average. The national polling average page reports the trendline adjustment at the bottom. On August 7th, the trendline adjustment was a total 2.7 points toward Clinton—certainly not 7.9 points.

edit: Here's more evidence

Holy shit. They're biasing in the wrong direction.

YouGov historically leans left according to 538's adjustments. They say that, on average, YouGov gives a 1.6 polling boost to Democrats.

Why, then, did they bias Trump's polls towards the Democrats? If they wanted to counteract the bias of the poll, it should have been towards the Republicans. That would mean that Trump would go from +2 to +3.6, not +2 to +1.

Is this true for every poll, or only dem-leaning ones?

They're adjusting all polls toward Hillary. That should be expected for Republican-leaning polls, but not for Dem-leaning.

Pollster bias isn't the only adjustment made in their model. It includes a likely voter screen, omitted third parties, trend lines, and allocation of undecided voters. State polls also include a demographic regression, and the polls-plus model includes economic data.

Or the bump was the "convention bump"

Lots of airtime and "celebs"

No, the convention bump is reflected in the raw poll numbers. What changed was the adjustment that 538 applies to every poll to correct for the "house effect" bias of each different pollster. Except this appears to have been an across the board adjustment of 8 toward Clinton, implying that every pollster has been producing polls biased towards Trump by 8 points. And that they only discovered this following the DNC. And that they didn't report what would otherwise be major story about the deficiencies of all national polling done this election cycle.

More importantly, it makes election fraud easier to swallow later.

I think the reason you're seeing the big disparity is if you're calculating these from nowcast then you have to take into effect all the other weights(which is impossible to figure out really or fact check for that matter).

If you're noticing it from strictly polls only and not polls+/nowcast.. then you need to take into factor the previous days polls they're using polls from up to a week ago, now those polls have less weight but trump wasn't doing well for many of them. The other thing you need to factor in is that STATE polls hold much more weight in terms of their forecast than NATIONAL polls.

For the polls only things they don't count the bounce, but she still was leading polls in general, if I'm not mistaken. You need to look at the polling data from the previous window that they state their using. It doesn't look at that surprising to me given that, but I'm not willing to dig. Each of the previous polls loses significance but say it was a leading poll from yesterday and the day before by hillary that's going to add to the weight. Is there a bias for hillary from 538? Probably. Would 538 keep it's so-called integrity? I don't know.

You haven't provided any proof to your claim op, poll averages change as new polls come in, that is why the numbers change. This is middle school math we are working with here.

You seem confused: the issue is not what new polls do to the average, the issue is retroactive and significant modification of old poll adjustments to match a very new change in weighting (uniformly in Hillary's favor) that has occurred without any apparent explanation.

You seem confused

I'm not the one posting a thread on r/conspiracy and doesn't understand the methology used by his source. 538 uses new poll numbers to adjust all of the averages based on their three tiered system. Oh course 538 has only guessed presidential elections correctly 99% of the time so the OP should send his research to a major news source. I'm sure OP could make a real name and a lot of money for themselves disproving 538's methology. It's only the most respected source in polling so I'm sure everyone would love to hear op's theories and "proof" of them screwing with the numbers.

I guess I should amend my earlier comment: you are definitely confused. Please try to wrap your mind around the content of the post before presuming to comment on it.

Prove me wrong.

Show us where 538 is skewing the numbers.

Show us where the most advanced polling aggregator got it wrong.

Go ahead and try your best, like I said early I'm sure every media source out there would love to see you disprove 538's methology.

Remember to show your work so we can take your research to the media for tomorrow's news cycle.

I can see the headlines now "Redditor Reveals 538's Polling Bias, Years of Research Disproven by Some Poster on r/conspiracy". lol

There's no need for me to prove you wrong: the original post already did. You just don't understand it - at all - and your problem does not appear to be correctable so I won't waste any more time on it.

That post didn't prove anything but the op doesn't know what they are talking about.

Still waiting for you guys to disprove the most well known the most accurate political polling site.

Right here.

Nope, still doesn't prove your point.

Still waiting for you to disprove the most well known and the most accurate political polling site.

Think of how famous you will be.

Sorry wrong link, I meant here.

That +1 shows that Trump is leading the YouGov poll taken on that date.

The 538 rankings show that over that pollster history they tend to favor democrats by 1.6%.

Now stay with me here, you do not add the two together and assume Donnie is leading by 2.6% overall ffs.

But if you don't believe why don't you try reading the directions on how to interpret the polls before you come up with some halfass idea and post about it on r/conspiracy.

http://fivethirtyeight.com/features/a-users-guide-to-fivethirtyeights-2016-general-election-forecast/

You're reading the chart backwards.

The +2 Trump is the raw poll result from YouGov.

The +1 Trump is the bias-corrected 538 version.

The bias is being corrected in the wrong direction. YouGov favors Clinton, but the bias-correction favors Clinton more.

That's the point. It isn't the magnitude of change; the direction is wrong.

Why don't you take this up with the media, I'm sure they would love to hear from the folks at Reddit conspiracy headquarters that 538 is incorrect and the armchair statisticians are here to make things right.

Uh huh. Well, all of that notwithstanding, the data show what I said, and your explanation doesn't explain it.

Nope, I'm going with the experts who vet 538 and prove on a daily basis they are correct in their statistical analysis, not a bunch of window lickers posting on r/conspiracy.

We can show you the original post. We can explain it to you. We can't understand it for you.

I read the original post.

The op is an idiot trying to disprove the best polling aggregator out there. But don't take my word for it just ask anybody else that understands high school math better than the op.

yeah - follow a herd and refuse to simply do the logic - the rational thinking. Once again - you'd be much more at home pasturing in /r/ShitAmericansSay instead of trying to fake your way through a thought.

Still no answer?

Why am I not surprised the experts at r/conspiracy cannot prove themselves correct when questioned.

You know who else predicts winners 99% of the time? Bookies that are in on the fix.

lol, everyone is against you little buddy.

That's ok you're in a safe space here.

Yeah - you're just the one not understanding anything that's been posted and challenging it like a pigeon on a chess-board. You'd be happier over in /r/ShitAmericansSay

lol, still waiting to hear of back from your statistical analysis experts?

Remember kids, if you can prove the 538 system wrong and they are skewing the numbers you could make millions with your own website "unskewing the polls".

Let me know when you get that website up and show us how right you are OP, lol.

It is posts like this that make people think this sub is just 14 years kids. Your candidate just really is losing, accept that.

Trump is losing in the polls. But the issue raised by this post is not the polls – which show a very substantial bump for Clinton that almost certainly goes beyond just a post-convention bump – it's the adjustments that 538 is making to the polls, and the dramatic change in the adjustments that occurred.

I just tweeted your info, hope you don't mind!

Not at all. Thanks.

"margin of error"

Move along, nothing to see here, ignore the man behind curtain.

Impressive insights into 538's tomfoolery. Every part of the media machine has to do its part to ensure that woman is elected. It's a sad election in so many ways.

FiveThirtyEight and DailyKos are paid promoters for Hillary Clinton masquerading as journalists.

[deleted]

When Trump wins,

lol, my sides...

Is this where donnie's ballcupers go to console themselves, kek.

You guys are so sad clinging on to this nonsense.

Yes? Someone has to be the first one to notice something. I have a PhD in political science, have taken quantitative methodology at the graduate level, and my dissertation did a lot with nonparametric inferential statistics. Nate Silver has a bachelor's degree.