Wow I don't post on here too often but I figured I'd post about my job switch since it's a pretty big change in my life. Specifically, I went from being a software engineer at DynamoDB in East Palo Alto to being a software engineer at MongoDB in New York City. I'm pretty happy with the change and I'll share my experience here!
I was working in DynamoDB for over two and a half years and I had a pretty good experience overall. I was working with many smart people and as a new grad there was everything to learn. And I definitely did learn a lot. But I think there were a few things that tipped me near the end of my time there which indicated that maybe moving to a new place would be a good idea.
My team in DynamoDB was extremely ops heavy and naturally so much time was spent on battling tickets. This isn't to say that we didn't work on features as much but definitely a lot of focus was aimed at operational work. Throughout my time there, I'd split the work as 80% ops and 20% development. I think this was a good learning opportunity because it not only required you to see how the system works overall but also look at faults in very specific areas of the system. Over time my knowledge of DynamoDB became stronger and I felt more comfortable working independently. What didn't change, however, is the quality control.
The biggest frustration I had was with the quality of not just the code but also how our team executed ideas and plans. I think my team did an amazing job in writing detailed technical documents and asking the right questions in design reviews. They were great at catching a lot of mistakes early on in the design process for any system being modified or created and they were helpful in pointing out alternatives and also brainstorming alternatives that could work.
What wasn't so great was the step after that. There was almost never any mention of code or implementation details. Or if there was it was too abstract to directly convert from document to code. Initially I thought this was due to me being a very junior engineer but I later found out that many of my peers were facing similar issues (both at my level and higher). Rather, it was all about timelines. You essentially break the project down into individual tasks and give an estimate as to how long each task would take an engineer - without ever discussing any implementation details. Tasks could be as vague as "implement integration tests". This mindset led to every project being over-promised and under-delivered and not a single project was done on-time correctly in my time there. And I slowly started to understand the differences between what customers see and how it works on the inside.
Code reviewers did not care much about best practices nor about the organization of code. As long as the core business logic was met and working correctly then it wasn't too difficult to get a 👍. This led to creating methods which were many hundreds of lines of code long and very deeply intertwined with other areas of the code - stuff that might not even be relevant or might never even be run. I tried to push back on some code reviews to follow best practices or to write unit tests. But then I got feedback that people saw me as a blocking point in the project rather than helping the team move forward.
I mention the quality control because this is how the team had been running pretty much since I had joined (and it indicates that this is the way things were running before too). The problem then comes when things need to be changed or upgraded. We never focused on keeping libraries up to date or working with the latest technologies. It was a very "if it works don't touch it" mindset. And I could somewhat agree and reason with that because DynamoDB is an extremely performant-focused database. Upgrades to internal libraries and what not requires tons of performance testing to see if those libraries can be used in production. But what didn't sit right with me is to never address it until needed. This created a very reactive environment rather than a proactive one and it destroyed developer velocity as well as work life balance. Our team followed a 24-hour shift for on-call rotation and it was extremely uncommon to not get alerted in the middle of the night.
In the prior 6 months to my leave a lot of things had changed. I got a new skip-level manager, many of my team members (as well as my sister team's members) had left (I think due to a mix of missed promotions as well as bad work-life balance) and overall team morale was pretty low. Because the number of people available was now low, people were constantly getting shuffled in projects which led to more delays. Engineers were complaining about work life balance and also saying that the team is too ops-heavy but I felt that the leadership team didn't care much about these concerns since there was too much work to be done. I give my manager huge props here for taking the step forward and arranging a few meetings to try and address some of the issues but nothing solid came from it unfortunately.
With many members leaving the team (and these were quite veteran people) I felt maybe I needed to take a similar path. I was also not alone in that "missed promotions" category. To be fully transparent I was denied my promotion at the end of the quarter while every other engineer at my level was promoted within my org. My teammate who joined a week before I did was promoted as well.
Overall, I was pretty devastated and I was quite pissed.
I maintained a work trackers document to track everything I had worked on over the past 2 years and compared it to what the other engineers had done to see if I had missed anything. I maintained very good relationships with them so they were kind enough to provide the information because they were confused as to why my promotion was denied as well. I asked a senior engineer that I was close to and he said that sometimes it's not always based on merit and achievements - something I didn't quite understand at the time.
After about 2-3 hours of comparing I realized I was wasting my time. Nothing I could say or do would affect the decision anyway and arguing it would just make me look bad. My manager, however, was helpful and did give me specific feedback as to what to focus on but that same fire I had in myself just went out (I think this got pretty bad because my manager even noticed I wasn't as enthusiastic towards work and talked to me about it). But my desire to go the extra mile for my work was gone. I delivered the bare minimum which was necessary over the next quarter.
I do see my promotion as a blessing in disguise because the day my manager told me I was denied my promotion was the day I was looking at alternative options.
Initially I was looking at different teams in DynamoDB because I still liked the product and the scale that I was working at but that eventually fell through. Partially it was due to the new members that had joined the leadership team, partially due to location (I was done with Bay Area) and partially due to the work life balance. It's not fun waking up at 4AM because of work.
I ended up firing up Leetcode (because unfortunately that's just how interviews are done) and started grinding and reviewing all the core algorithms / data structure concepts. I was also glad one of my other friends from college was looking for a new job as well and we talked to each other about problems and helped each other in the interview process on not just algorithms but design as well. That definitely helped out a lot.
After about 3-4 months later I finally secured my new job at MongoDB and I couldn't be more excited. I had read a lot of MongoDB's blogs and looked into NYC as well. I think it's an exciting city to be in so I'll see how that goes (currently I am working remote due to the pandemic but I will move there soon). I also like their approach towards building software and having a very engineering-oriented mindset. Their emphasis on culture is huge whereas it was a bit non-existent in AWS so that's another aspect to look forward to.
I mentioned a lot of negative points about DynamoDB but I can safely say that working there was well worth the experience. I really liked my team members and my manager was amazing. She was probably the reason why I stuck around a lot longer than I thought I would. I found the timing to be funny because I announced my resignation on the day she said my promotion went through. On top of that both my manager as well as Jeff Bezos announced their leave as well!
But the problems I worked on in DynamoDB are very unique and I don't think it's easy to find those kinds of issues in other software environments simply due to the sheer scale that DynamoDB/AWS operates at. There was a ton of learning and I think it definitely gave me a huge kick start to my career. I would say that if you are willing to take a hit to your work life balance then DynamoDB (or any mature product in AWS in general) is a great way to go through a ton of learning.
Hoping that MongoDB can be a great experience for me as well!