The Federal Reserve Destroyed Our Site

Dear Economic Populist Readers.  You may have noticed we have stopped publishing articles.  This is because the Saint Louis Federal Reserve Fred Graph system was changed without even a warning.  They removed the very fast, professional economic graphing system and API, which we used heavily, and replaced it with more of a toy.  As a result over 3,000 articles on this site are now ruined and have terrible, distorted graphs that no one would consider publishing professionally.  Worse, the site articles now have very slow page loads for each article utilizing the old FRED graphing system and some take over 30 seconds to render, if at all.  We also are throwing many web errors due to the newly broken FRED calling API.

Even most disconcerting, we warned St. Louis Federal Reserve this would happen and were promised the old, professional graphing system and API would remain in place for legacy use.  We showed them web developer test results, pointed out the many issues with their plan and slow technology, to no avail.  In essence, we were blown off.

Now the site is destroyed.  We're terribly sorry.  We were told ages ago the API would not change and the way we using FRED was acceptable.  Even though our technical expertise was ignored, we were promised by the St. Louis Federal Reserve they would not change the graphing system for existing graphs.  Thus, we were caught off guard with way too many articles utilizing FRED.

FRED was a free public service offered by the Federal Reserve, and we were most grateful for it.  Fred used to be the most professional graphing system on the web and was a great resource for many economic sites and economists, not just ours.  One could create professional graphs, easily access most economic data series, code, customize and calculate new graphs, on the fly.  We utilized FRED because there was no other way to generate graphs to illustrate our economic observations that quickly.  For us to generate optimized images that quickly, the only other option was to build such a system ourselves.  The old FRED enabled us to publish articles with illustrations in record time.  With the old FRED system, we could generate new graphs in under one minute that loaded in milliseconds from the Saint Louis Reserve servers.

Now it is gone, destroyed, blown away and so is our site.

For all of our readers who say never trust the Fed, well, turns out you were right.  This is also a violation of engineering development 101.  Never break legacy APIs, never leave your customers hanging out to dry, yet unfortunately this is what has happened.

One can say what do you want for free and normally I would concur, yet so often we were assured the Saint Louis Reserve team would not break the API, would not destroy legacy graphs and the professional, optimized graphing system.  At least Microsoft has warned they are leaving XP users high and dry even though there is absolutely no reason to stop support for this operating system.  In our case we were promised the opposite, that the Professional FRED graphing system would still be in place for legacy graphs and links.  We have complained, we have pointed to data, we have called, all to no avail.  We're simply left high and dry without a care in the world and since FRED was a free service, there is not a thing we can do about it.

Dear readers, let us know what you want us to do.  There is no doubt the site must be taken down, the entire database modified, alerts to search engines that over 60% of our content is now destroyed, not professional with loading timeouts is required.  We apologize for we know this has already caused sites which link to us enormous problems.  The removal of our site content will break thousands of links around the web.  We will lose our professional status as a news source.  We know this will cause site traffic to drop, page rankings to plummet and readers to be lost.  In fact it is already happened to our site.  We are being dropped from search engines almost immediately due to the terrible graphs displayed instead of the old, fast loading Professional graphs shown on the site previously.

In essence, almost from the ground up, we have to start over and in an emergency situation, redesign the Economic Populist plus remove over 60% of our previously published articles.

Let this be fair warning, dear readers.  Trusting others, their APIs and methods to display content around the web is clearly a huge mistake.  Never trust the Fed or any other API which has the potential to destroy a site if broken will be our motto if we rebuild The Economic Populist.

Thanks for your patronage.

The Economic Populist
January 2008 to March 2014
RIP

Update April 4th 2014:

Many people have requested we keep the site going.  The FRED graphing system destroyed our site with the over 3,000 existing graphs we displayed changed on us without notice,.  They slow down the page load to the point an article will not display at all, they changed the colors, size and resolution of the graphs so they are unreadable..  We are also without a graphing system now.  There is no way for us to produce so many graphs amplifying our economic statistics analysis.  All we can say at this point is consider emailing and calling the Saint Louis Federal Reserve and voice your complaint.  Honestly, the site is not fixable.  We'll have to delete over 3,000 articles that we spent many hours writing to fix the site.

Subject Meta: 

Forum Categories: 

Well, that has to hurt a lot.

Don't really know what it would take, but I this doesn't stop you.

it is very upsetting

quite a betrayal.

don't give up, I like your site

I hope you don't throw in the towel because I appreciate your site.

In the future, I suggest you post your FRED charts as images loaded to your website instead of as embedded links to FRED. That way your blogs are safe even if the FRED links fail.

FRED is toast

There are a good 7,000 image links and the reason was it's a cheap CDN and by code I could update/make new graphs in < 1 second. So, either I create our own graphs from data from other sites,say the BLS, Census, or I write my own code to translate Excel data into images.

If I had know they would have done this, I would have written a script to capture all 7,000 images but it's too late now. Honestly, this is almost malicious.

The FRED tool wasn't free, it was public

It was provided by a quasi government public utility. It arguably ought to be held to public standards of accountability and transparency. And, arguably, is operating with taxpayer funding.

other government agencies

Nothing like data hiding through technology. See pretty (not, actually horrific resolution, not even progressive) graph, oh look, it has a slider, bummer it tells you nothing now about the state of the economy in one image, unlike before.

What can one do here, if someone pays me to write up a system, I could probably do it in six months but FRED was really sophisticated and to put a package like that together, really is a full-time development job.

I see charts!

I still see charts in your articles... Your post seems overly harsh to me. I have called FRED to troubleshoot issues with their excel add-in before and they were very helpful to me.

FRED graph was not free. It

FRED graph was not free. It was paid for my the population.

i'm in the same boat..

because i had all my FRED graphs linked back to my saved graphs on their website, and they did a real job on my graphs with multiple data sets, like the one i used 8 days ago: https://research.stlouisfed.org/fred2/graph/?graph_id=165654&category_id=0 ...on my blog, the 12 data set names in the strip across the top consume almost all the pixels i had allocated, leaving virtually no graph...

it took me about 8 or 9 tries to get one usable graph to go with my CPI coverage this weekend...one i often use, which resets the CPI components to 2000 so the various indexes could be compared apples to apples, was damaged beyond repair, so i tried to recreate it, and every time i tried to save it (here's a screenshot of what i wanted) they changed the base observation date to 2008....turns out you can only reset indexes to the beginning or end of a recession, so here's a picture of what i ended up with...if you go to my saved draft at FRED on that ( https://research.stlouisfed.org/fred2/graph/?graph_id=167726 ), you'll see the interactive shows the beginning of the 2001 recession as "Thursday March 1st"; in fact all reports in FRED are all now dated with the day of the week the first of the month falls on, so obviously those graphs show some monthly reports being released on a Sunday... leaves me with the feeling they hired the same developers that worked on obamacare..

as you may recall, i'm using a bastardized version of IE 8 loaded with tools i use for my weekly links blogging, and the new FRED doesnt even display on that (i have an email from a senior developer confirming that bug)...but i checked with my correspondents, and so far they all can view the new graphs, as can i using Chrome, so i'll probably try to use them...even that i have to go easy on, because having five of them open at once really slowed down my chrome browser..

rjs

Exactly, sorry rjs

The graphs are just terrible, gone is the high resolution so I can show, clearly what I am pointing out in the text.

But due to the overwhelming response, folks, I wll fix this site. Gone is all of the great eye candy until I can build a solution, although our custom graphs I will still try to create.

So, we're going to have to tough it out, I will have to take the site off line, heavy modifications and such,
but I will keep EP going.

I wonder how many others are flipping out over this, their sites ruined. Looks like anyone who linked to any graph?

Robert, where is your blog

Robert, where is your blog broken? I see nothing wrong.

u kidding?

This site has over 3,000 articles linking to old FRED graphs, perhaps you are not reading the details to see the distortions or noticing the very slow page load. We're better than others but it's still quite the disaster.

Our site page load speed was under 2 seconds, any post, That's a professional site page load, now it is 5 seconds and worse. That's not acceptable as a news source and our load is super light.

Our hits went from 4k a day with 15k reader spikes to ?? Well this is getting interest but still, 1.5k a day? Yesterday it was down to 200 hits. that's simply noise for readership.

you're still faster than most

i dont notice a slower load here, while my encumbered browser takes forever on pages from huff post, bloomberg, guardian and especially business insider

i did notice the somewhat distorted graphs on your IP post earlier this week, thought that you were going with it the way it was..

rjs

hmmmm, something just changed!!!

ha ha ha, something has changed and it is for the better, no wonder people are saying they see no difference.

We'll see, but maybe my public rant actually helped here, I have to do more testing and see but if true, then the issue is just the new graphs.

Basically a page which had a page load speed of 5.84s is now down to 2.88. Now that is not 1.2s which is what it used to be, but for legacy, not in Google news, I can live with 2.88s.

I still have the issue of my super mega fast graphing solution is destroyed but I sure hope they reverted to some of the old engine which would at least "save" in part the 3,000 existing articles.

Will update if true, have to take the site off line and do some testing.

I cannot notice a difference.

I cannot notice a difference. And saying this would have destroyed the site is a wild exageration.

By the way, hit counts should not be influenced by this anyway. Evn if pictures take some time to load, the page is still loaded first.

FRED looking for a web developer

seriously: https://research.stlouisfed.org/jobopps/developer.html

interest in economics a plus...

rjs

too funny, well, we know how I would do already

I even offered to volunteer, which if anyone knows me in "real life", I'm so overqualified and experienced that was quite the gift, yet that too was completely blown off.

Hilarious considering my consulting rate. ;)

I'm getting some solutions going, hopefully will take EP off line this evening to start the 1st phase of the "repair". Going to have to do this in stages and my "real life" I have critical tasks that I must address on top of this disaster.

I promise, EP will survive, we'll keep it going, this is just a critical wound that we have to repair and in terms of offering extensive economic eye candy, that solution is going to take a long time. Now that is real code, real architecture, so honestly I just don't know what i can throw together on it, although I am aware of various open source components that will work as part of a solution, Huge TBD.

worse than i thought

the markings across the bottom of my GDP bar graph indicate Q4 2012 for the first quarter of 2012, and Q2 2012 for the third quarter of 2012 (yes, the yearly dates are backwards) and it also includes bars for Q4 2014...then the interactive slider on my income and outlays graph allows me daily readings for PCE and DPI...

edit: i also see the end date for the data on that graph is marked 2019-08-06

rjs

rule # 2650 of web design

always have a display for no javascript and always use localized JQuery and not javascript but what the hell.

Anywho, I hope to get the first stage of our site fix this weekend. I'm having an "elder crisis" on top of things and I must absolutely write about that in terms of the economics.

I've had some other geeks who read EP start to email offer to volunteer so maybe between us we can throw together a graphing system. the real problem i see is obtaining easily the actual tabulated data to input into the graphing system. That too is gov. data and it seems disallowing, denying access of data has been going on at other agencies. I think part of that is security, part budget cuts but the reason, hmmm....

At least we're not the only site who is up in arms about what is going on. This really chokes us providing Journalistic level of coverage on economics.

Data

So why is it the government wouldn't want people to have historical data?

Calling All archivers, image hunters and caching

I finally got to phase one of the site fix. This is going to be slow going and anyone watching, you're probably seeing the FRED disaster get worse and worse, no doubt they are also giving us the finger after the final very public call out, but hey, they ruined our site! Gone is the awesome free factual public analysis!

Anywho, next phase is to disconnect the St. Louis Fed from this site, so I'm looking for replacement graphs. I'm not sure how exactly I am going to do this, I need to do it before publishing new content and I was hoping not to break 3,000 links by destroying 3,000 articles that link to FRED for graphs.

Obviously that cannot be done manually, unless someone has 5 years, this is how long it took to write these.

FRED work in progess

seems everytime i check my old posts the graphs have changed; they must be applying changes on the fly without testing them...the latest is that most, if not all, of my bar graphs have been turned into fairly meaningless line graphs, while my explanations of what the bar graphs had meant remains in place..

rjs

Losing FRED

Assuming you can't prevail on the St. Louis Fed to return to the old graphics platform, please do whatever you can to render your most important published grafs in an alternate system. This site is too good for me to be indifferent about it shutting down.

every week it's something new

this week FRED changed the font size on the headers from 13 to 8; the former was too large for graphs with multiple data sets, but these are barely readable: https://research.stlouisfed.org/fred2/graph/?graph_id=125150&category_id=0

rjs

It's unbelievable.

I sympathize. I loved FRED. Now it's worthless. Why aren't more bloggers complaining about this???

My first visit. I came here by tracking down a "Democracy in America" excerpt on inheritance laws, Numerian's post from 2010. Thanks!

It's a shame what they've

It's a shame what they've done. I'm looking forward to having Econ-Pop back up at some point as it was my most trusted read on the entire internet for any sort of news. I'll keep checking periodically to see how long it takes to get your site back up.

The Worst Thing I could do is submit anything with FRED Links

Really, I was hoping to publish something large in size with FRED Graphic. I am hearing that that is a terrible idea to have any content from FRED even if I do not use FRED links?

Burton Leed

FRED

The St. Louis Fed is technically illiterate and you may be surprised just how callous toward the idea of public service and customer service.

Milton Friedman

July 30, 1998
hoover digest » 1998 no. 3 » asia
The Hong Kong Experiment
by Milton Friedman.

Cowperthwaite was a Scotsman and very much a disciple of Adam Smith. At the time, while Britain was moving to a socialist and welfare state, Cowperthwaite insisted that Hong Kong practice laissez-faire. He refused to impose any tariffs. He insisted on keeping taxes down."
"I met Cowperthwaite in 1963 on my next visit to Hong Kong. I remember asking him about the paucity of statistics. He answered, “If I let them compute those statistics, they’ll want to use them for planning.’’ How wise!"