It is common and normal for Redshift users to experience significant performance and behaviour problems. This occurs because Redshift is a complex and very knowledge intensive database, but where meaningful information about how to operate Redshift is wholly lacking - as far as I know, this site, my site, is all there is.
The reason I know about Redshift is that I am 51 years old, have an extensive background in databases and have spent the last six years, full-time, investigating Redshift by directly experimenting on the database and finding out what it actually does. I myself have researched and discovered the knowledge which is needed to operate Redshift correctly, and it's available no-where else.
The value proposition then is simple : I know Redshift a light year better than anyone else, I really do know it back to front, inside and out and from side to side, and so I know why Redshift is going wrong, I know if it the right or the wrong choice, and where I am not AWS, I will tell you when it is not appropriate to use Redshift, and I can tell you what database technology you should be using.
Usually, clients present in the midst of crisis. They have a business to run, they're using Redshift, and the systems they have which are based on Redshift are performing so poorly that the critical business functions are no longer viable. The first and pressing task then is getting these systems back on their feet enough, and as quickly as possible, that these critical business functions return to viability and the so business can for now continue and we then there is time to look at what went wrong, why, and how to ensure it never happens again.
So, once the crisis has passed, there is then discussion and explanation about Redshift, in the light of the client's use case(s), to determine if Redshift is the correct choice, and if so, why it was going so badly wrong, and if not, why not. In truth, with every client I've worked with so far, Redshift has not been the correct choice, and I have advised clients to move to either Postgres, Postgres-XL, or Snowflake - and they have done so and this has solved all their problems - but migrating database, especially when considerable work has been invested in existing Redshift system, requires absolute certainly that this work must be done, and that the migration will be successful. That's something I provide.
As for the crisis, usual causes are one or more (the more, the worse the crisis) of;
My name is Max Ganz II.
As of 2023, I am 50 years old; so I've seen quite a lot, and I've done quite a lot. I am currently based in the UK (but hold EU and UK passports). When needed, I relocate to clients for the duration of a contract.
I began using Redshift the day it came out - I was in a three man startup, and we were trying to get into the beta programme - I then worked with Redshift intensively, 80 hours a week, for a year and a half.
As an early, heavy user, I ended up having one-to-one team meetings with one of the Redshift team managers, and one or two of the features now in Redshift come from my suggestions in those meetings - the ALL distribution style, for example.
Fast-forward to mid-2017, and I began to investigate Redshift full-time with a view to writing a book, which pivoted, becoming this web-site and Combobulator, a Redshift management AMI, and so since mid-2017, apart from the occasional contract, I have been investigating Redshift full-time.
On this web-site, I publish the unique information generated by these investigations in the white papers section, and maintain and publish the results of a range of long term Redshift monitoring efforts, including what are to my knowledge not only the cross-node type benchmarks, but also cross-region benchmarks (and it turns out Redshift performance varies greatly by region).
As for me, I began programming when I was 11 years old. I moved to C when I was 18, and I am and always have been a C programmer. I've done Windows kernel programming, written my own database from scratch in C, and I have another project where I publish and maintain an open source library of advanced data structures. I know computers from the hardware upwards.
Pricing depends on the work and the client.
Initial meeting (video conference app of your choice) is free, so we can get an idea of what's involved and pricing.
Regarding payment, I have a multi-currency FinTech bank account, so what happens is my business issues you, or your business, an invoice, which you pay using the usual bank transfer system in your country. That's all you do; it's a invoice, nothing more, and you pay it as you normally would pay any invoice. No additional paperwork, no responsibilities for taxes at my end, nothing like that at all.
Prior to formally publishing this page, companies occasionally approached me for Redshift consultation. These are the recent significant such events, which provide a useful couple of references and reviews for getting started.
Barring the client's wish not to be listed (in which case they will be down as "anonymous"), all work is listed, has the company web-site, contact details plus LinkedIn page for the source(s) of the review(s), dates, the reviews(s) and my description of the work and what happened. To the greatest extent possible (I could just omit clients and you wouldn't know), you get the good, the bad, and the ugly, which you can verify for yourself.
2022-08, Starred B.V., The Netherlands, Bram van Gestel, CTO (email, LinkedIn)
In 2022 we were running a Redshift migration project for our BI solution and we experienced performance issues. With Max’s guidance and knowledge we come to the conclusion that Redshift was not a good fit for our use-case, due to it underlaying architecture.
I would describe Max as a very knowledgeable person, and the collaboration was as professional and pleasant.
The client initially was experiencing serious problems with Redshift, in particular with slow interactive/dashboard performance, enough to lead them to look for the resolution of these problems prior to an upcoming release of their service. We did we what could, which in this case in particular involved hand-picking column encodings for the major tables and adjusting WLM configuration, and I conveyed to the client that Redshift compiles its queries, and so there are irreducible minimum query latencies, which makes Redshift always and unavoidably inappropriate for interactive use, and in due course, the client undertook the move to Postgres, where they are now, and they report this move worked well.
2022-12, EdOps Inc., United States, Josh Marks, CTO, Co-Founder (email, LinkedIn)
We feel extremely grateful for the deep support of Max @ RedShiftResearchProject.
Redshift is an extremely difficult beast to tame and the behavior of our instance was often inscrutable. We feel extremely grateful to have worked with Max, Redshift whisperer, who was able to quickly diagnose problems and prioritize the steps to vastly improve performance. He has deep technical knowledge of the product, a warm comical wit, and can challenge Dos Equis' most interesting man. I highly recommend that you work with him. Feel free to contact me.
The client initially was experiencing severe problems with Redshift, such that regular ETL processing was taking long enough that it would still be executing by the time the next run of that process began. We put out the fires, which in particular involved Metabase behaving in an acutely hostile manner, where this work resolving very harmful but essentially superficial issues.
In due course, we came to look at the next set of issues, and these were more fundamental and intrinsic to Redshift, not matters which could be avoided or fixed, but which could only be solved by operating Redshift within its limits and according to its design, which indicated Redshift was not an appropriate database technology for the use case in hand, and the client after testing alternatives moved to Snowflake.
2023-03, Railsware Solutions FZ-LLC, Ukraine, Anatoliy Yevpack, Associate Director and Technical Lead, (email, LinkedIn)
We are extremely grateful for the assistance that Max has provided us as a consultant. He shared numerous insights into the internals of Redshift, which quickly proved to be invaluable and unveiled the actual problem we were facing. Gaining an understanding of the underlying reasons behind the issues we encountered led us to realize that we were employing the wrong tool for the task. Consequently, we made the decision to transition away from Redshift. This not only saved us a substantial amount of money but also spared us from those issues. We should have reached out to Max much sooner!
The client was in the design stage, and so I was able to fully inform them about what they'd get with Redshift, and they and correctly for their use case opted not to use Redshift.
2024-03, Prismforce Inc., India/USA, Amit Singh, Senior Technical Architect (LinkedIn)
Max has deep knowledge on internals of Redshift and has helped us on understanding the entire workflow. We were in an ambiguous situation to choose AWS Redshift, but based on Max's concrete case studies we were able to make an accurate decision. He also helped us with tooling for our analytics use case and we are happy to make the decision. Thank you Max, and we would refer you if some of my acquaintance is exploring AWS Redshift.
The client was looking to move their existing data and BI system to Redshift, and had developed a POC. After consultation, it was wholly apparent Redshift was not suitable for their use case. I advised the client to examine PostgresXL and Snowflake.
You'll receive a confirmation email. I'm in the EU ATM, so if you send when I'm awake, you'll likely get a response pretty quickly - few hours, tops.
Note all and any information you enter is used and only used to reply to you. You are not added to mailing lists, aggregated, sold on the other companies, or anything like that.
Home 3D Друк Blog Bring-Up Times Consultancy Cross-Region Benchmarks Email Forums IRC Mailing Lists Reddit Redshift Price Tracker Redshift Version Tracker Redshift Workbench System Table Tracker The Known Universe Twitter White Papers