freistil News

Three findings on my flight home:

• Compared with Hungarian, German with its puny three umlauts can pack up and go home.
• Compared with Ireland, Hungary has much nicer weather.
• Compared with the pubs at home, in Hungary you can treat four or five times as many people to a beer for the same amount of money.

As you can see, Hungary has the advantage in many regards and I had a great time here at the Drupal Developer Days this past week.

When I arrived in Szeged on Wednesday evening, Drupal 8 coding sprints had already been running for a few days and they’d continue all week. During this time, up to 150 Drupal developers were working to make progress on code and documentation issues. There were 115 commits to Drupal core and after removing 19 blocker issues, we’re now 40% closer to Drupal 8 Beta. Our Git repository traffic was so high that it even triggered drupal.org’s DDoS defenses!

At the event location only, we consumed 5500 sandwiches, 1000 servings of coffee, 240l of beer, 300kg of sweet snacks and 120kg of bananas. The venue was a really good choice. It had all the space we needed, good catering and the WiFi worked well throughout the event. And having the Novotel (with its amazing value for money) right next door is unbeatable convenience.

From Thursday to Saturday, there were a lot of interesting presentations as well as a reprise of the “Caching Deep Dive” multi-hour workshop that had a lot of success at DrupalCon Prague. Almost every talk referenced the upcoming Drupal version. And although many things are still in flux, it feels to me like Drupal 8 is taking shape.

What drove engagement most was the great community spirit. Everyone was welcome, from the Drupal novice to the long-time core contributor. There were smiling faces all around and you could simply walk up to anyone to have a chat or ask a question. People got together spontaneously, be it to code or to go have dinner. These personal experiences are what I love most about the Drupal community. If you haven’t been to a DrupalCamp yet, go to DrupiCal now and see what’s happening near your place! Go on, I’ll wait here.

My personal highlight was the #AberdeenFreistilCloudBox party on Friday night which I had the pleasure to co-organize. I had contacted Aaron Porter before the event, suggesting we join forces and do something about the community’s lack of awareness of our European Drupal hosting companies that we both had perceived at DrupalCamp London a few weeks earlier. When I met Aaron on Thursday morning, he invited me to join him in scouting for a party venue for Friday night. We looked at two bars in the center of Szeged and decided on hosting (that’s what we’re good at, after all!) the party at the CoolTour Cafe. We were able to make a deal that secured our guests 100 free beers as well as free admission to the concert room where one of Hungary’s best-known Jazz singers was going to be on stage. And when on Friday afternoon the number of sign-ups for the party crossed the 100 mark, Aaron and I decided to throw in another 100 beer vouchers. The party was a success and fun was had by everyone. The title of “drink distributor extraordinaire” goes to Dave Hall who’s apparently related to a huge Dutch family since he returned every few minutes to grab a voucher for another member of the Needabeer clan.

In the end, I’d like to congratulate the organization team around Kristof van Tomme to a great Drupal community event! I’m thankful for a lot of good conversations, and happy that I overcame my initial reluctance to register and got to be a part of Drupal Developer Days Szeged 2014.

• Read more

2014 is the year of Drupal 8 and we’re getting our freistilbox hosting platform ready to host its first Drupal 8 websites. Of course, Drupal 8 will be the central topic of the Drupal Developer Days in Szeged from 24 to 30 March. We can’t miss such an important event, so Jochen will be following DrupalMarvin’s invitiation from DrupalCon Prague last year.

Jochen is going to be in Szeged from Wednesday the 26th to Sunday the 30th, so if you’re there, make sure to say Hi! He’ll be more than happy to get you a beer or coffee, and you can ask him whatever you’d like to know about freistilbox.

And don’t forget to pack your towel!

• Read more

February put an end to the quiet time at the beginning of the year. We went into our final preparations for moving our business to our Ireland-based company freistil IT Ltd. From April on, all our services will be provided – without any changes in quality or technology – by our new company. We’re very excited about this new phase in our company’s development!

We kickstarted our community engagement by visiting DrupalCamp London. With about 600 attendees, it was an impressively big event for a local DrupalCamp. Discussions at the CxO meet-up and conversations during the whole event made us realise that we need to ramp up our marketing efforts because most people seem to know only the two US providers for Drupal hosting. We think this needs to change. ;-)

Taking the opportunity of Markus and me being at the same place, we stayed for another two days after DrupalCamp for our Q1 board meeting. Inspired by Marc Benioff’s book “Behind the Cloud”, we worked on defining our company’s key vision, values, methods, obstacles and metrics. I’m going to write some more about that at a later time. Culture is an important part of business and these basic aspects will be the foundation on which we’re going to grow our company.###freistilbox

Our hosting platform keeps growing steadily, with now 266 websites running on freistilbox (January: 233, +14%). We’re happy to see our availability improving as Pingdom reports a total uptime of 99,97% (January: 99,94%). Last month, we expected our traffic to cross the 10 TB mark, and it did so with an impressive 30% jump from 9.6 TB to 12.5 TB!

###Help Center

Probably mostly because of the short month, our technical support numbers are a bit lower than in January. We received 149 new support requests (January: 181) and resolved 127 (January: 140). The ticket backlog was 35 (January: 41).

This month, our average resolution time came in at 22.6h, nowhere near the insane 250h in January. We managed to answer 43% of new tickets within an hour (Jan: 45%) and another 23% within 8h (Jan: 28%). 15% of new requests were answered within 24h (Jan: 8%) and the remaining 20% took us longer.

We’re glad to report that our customers’ support satisfaction remained at 100%.

###Operations

In February, we grew the number of servers to 282 (Jan: 264, +7%). We’re now collecting 107,706 metrics in our monitoring system (Jan: 99,518). The number of on-call alerts went down to 1397 (Jan: 1781, –27%) and we’re working on improvements that will lower this number even more significantly.

###Other notable stuff

In February, I celebrated my 44th birthday. It was my first birthday party in Ireland. It feels good to get settled in, and having freistil IT based in Ireland now, too, will make me feel at home here even more. I’m very excited to see how my life and work will develop over the coming months!

• Read more

Tomorrow, I’ll fly to our neighbor island for DrupalCamp London and Markus is going to join me on Friday. Together, we’re going to breathe some community air again and get a feel for what British Drupal shops need in terms of hosting.

With 600 attendees, DrupalCamp London is going to be an impressive event! There will be 30 community sessions as well as BoFs and sprints across the weekend. I’ll certainly try to at least attend the “Next Generation DevOps” and “Concurrent Programming” talks. I’m also looking forward to the Drupal CxO meet-up on Friday before the actual conference.

For Markus and me, it’s a valuable opportunity that we’ll be at the same place at the same time for a change. That’s why we’ll stay a few more days more after the weekend to do important strategy work for 2014 together.

We’re very excited to meet a lot of enthusiastic Drupal developers In London! So, if you’d like to join us for a pint and talk about your Drupal hosting needs, simply drop us a line via email or on Twitter!

• Read more

Do you still remember these text messages you should not have sent last night? SMS von gestern Nacht (“last night’s text message”) does! The entertainment website publishes texts from its users that are simply too good, funny or rad to be read only by their original recipient.

SMSvonGesternNacht.de has been running on freistilbox since 2013. We thank the founding team for their following feedback:

Why we chose freistilbox

Our first contact with freistil IT was already in 2010 when they still offered their hosting under the name “DrupalCONCEPT”. We were looking for good hosting solutions for Drupal and had found freistil. Back then, we decided on a single managed server from another provider. Over the years, we extended it to a group of three servers. When we planned a relaunch in 2013, the Drupal experts from Palasthotel recommended a switch to the freistilbox hosting platform that they praised enthusiasticly.

Out of experience, we wanted to build on a proven and reliable solution. To learn the detail aspects of freistilbox, we invited Markus from freistil IT to our office in Berlin. In this meeting, we worked out that freistilbox would cover our needs very well. Its caching infrastructure, optimised for Drupal, promised high delivery performance for our many anonymous visitors. At the same time, the price was very attractive; we don’t pay significantly more for freistilbox today than we did previously for our managed servers. freistilbox brings additional advantages, too, for example the developer-friendly design and the fast technical support. The easy scalability is another important factor because our traffic grows steadily.

Our hosting setup

At launch, we started with a cluster of two freistilbox S. Since performance is more important to us than availability, we decided after a short time to switch to a single freistilbox M.

The launch

Our start with freistilbox was a very good experience. With one exception, everything worked out of the box. And the problem that our voting feature at first didn’t work because of the necessary extra cookie was quickly resolved by the freistil team with a custom cache configuration. The target-oriented way we discussed and solved this problem together with Palasthotel in a “war room” chat conducted by freistil IT significantly increased our trust in the team’s competence.

Daily operation

Hosting on freistilbox is fun because we don’t have to take care of anything. freistilbox simply works and we notice little of daily operations. Our questions usually are answered in much less than 24h and contact with the freistil IT technical support always is a pleasant experience.

If there is a problem eventually, someone immediately deals with it. At the end of 2013, our website suffered from network issues, today our availability is 99,9% even without redundant servers.

Since the launch on freistilbox, our traffic has more than doubled from about 18 million to 40 million page impressions per month. The highest traffic spike has been 1.5 million PI on a single day. The website still runs with full performance without us having to grow our configuration. We’re now considering allowing user logins which would probably require an expansion.

freistilbox leaves little to be desired. Comprehensive statistics that show how well our cluster is working would be a great additional feature, though.

freistilbox clearly holds up to its promise “work efficiently, sleep peacefully”. When the next vacation comes up, we’ll be able to go on holidays without worrying about our hosting.

• Read more

Starting with January, we’re going to post a monthly summary that will give you a bit of insight into what’s happened at freistil IT.

freistilbox

On our managed hosting platform, we’re now hosting 233 websites for which Pingdom reported an average availability of 99.94%. When we take into account that some of our customers chose to run their sites on a single box and/or near the maximum capacity of their cluster, this is a good result.

Our edge routers delivered a total traffic volume of 9.611 TB in January. Let’s see if we’ll crack the 10 TB threshold in February!

Help Center

In January, we received 181 support requests. We solved 140, leaving a backlog of 41 tickets (most of these are usually pending customer response). Our average reaction time in this month is… catastrophic. A single request that went unanswered for half a year because it was covered elsewhere launched our average response time to a whopping 250 hours. When we break reaction time down into categories, you can see that this is an absolute outlier. Of all support requests in January, we answered:

• 45% within the first hour,
• 28% in 1–8h,
• 8% in 8–24h, and
• for only 19%, it took us longer than 24h to react.

We’re very happy that the support feedback we got was 100% positive. Here are some notable comments:

• “Competent feedback as always.”
• “Super!”
• “Fast and reliable!”
• “freistil is always there when you need them.”

Operations

Our team is taking care of 264 servers, resulting in an average of 132 servers per system administrator. We think that’s quite respectable.

Each minute, our monitoring system processes 99518 separate metrics. In January, it sent 1781 alerts to the on-call engineer. This is an enormous amount, 54% more than in December. There were no significant outages during this month, though. Like with our support request reaction time, this value is so high because of an anomaly for which we needed to adapt our alert thresholds first.

Community engagement

Jochen attended the first Drupal Dublin meet-up in the new year. The group was treated to some detailed peeks behind the curtains of a huge museum website.

Other notable things

We signed the contract with our first employee! It took us quite some time to work out the right way to put our Results-Only Work Environment into legal writing, but now we’re excited for the first system administrator to join our team.

Overall, it’s been a good start into 2014 and we’re highly motivated to make it a great year!

Any questions? Please post them in the comments!

• Read more

2014 has gotten off to a good start but it’s still time to do a review of the last year. Learning is a big part of what we do at freistil IT and 2013 did teach us quite a lot of things!

Events

I don’t exaggerate when I say that 2013 started with the most important events of the whole year. My son Richard was born in January and Markus became a #newdad with the arrival of his wonderful daughter Marlene in February. First there had been a comfortable span of time between the two estimated birth dates, but Richard decided to take his time while Marlene wasn’t patient at all. This lead to the first challenge in 2013: How can we both at the same time put our “Family first” principle to practice by taking care of mothers and children without losing our productivity (or even actual business)? The serious product problems we had at that time made things even more difficult, but I’ll come back to that later. Well, we managed somehow and two beautiful children are celebrating their first birthdays these days.

In June, we went to the Emerald Isle for Drupal Dev Days Dublin[1]. freistil IT was Gold Sponsor and we almost didn’t get our freistilbox banners made in time because of a botched job by the printer. At the event, we launched freistilbox Solo, our virtual freistilbox environment for development and testing. I had arrived a few days early to start scouting for houses because this would be the year my family would finally make a long-held dream come true and move to Ireland.

In August, I flew over with my family and, as luck would have it, we found a house on the very first day of hunting. freistil IT is a distributed team and completely location-independent, so my move didn’t cause any major disruption. I only had to make do with a 3G modem until they switched on our broadband. In Freiburg, I had worked from a shared office in town, now my office is only a few steps from my bedroom. A home office has its own challenges, but if things gets too distracting, I simply go to a coffee shop like the one I’m typing this right now.

Drupal Ireland is a lovely bunch of people and I enjoy the monthly meetups in Dublin. In September, I met a few of the folks at the Prague airport where I had arrived for DrupalCon Europe and not much later, we found ourselves at an Irish Pub watching the GAA final. Prague actually was my first trip to eastern Europe and I enjoyed it very much. DrupalCon was a good opportunity to catch up with some of our customers. Not all of the feedback we got in these talks was positive but that’s not why we do them anyway: Improving first needs learning what we can do better. That’s why we appreciate honest feedback , and in the long run, constructive criticism pays for both sides.

We attended not only Drupal conferences, though. Since we’re doing cutting-edge IT work, we spent time exchanging knowledge and experience at DevOps Days in London and Berlin as well as at the Open Source Datacenter Conference in Nuremberg. We found these events great to gain new impulses and to share what we’ve learned.

During the year, especially in the second half, the reliability of our freistilbox hosting platform suffered a lot from network issues. In order to be more resilient against power or network outages, we had spread our servers over several datacenters. We didn’t experience a single outage on the datacenter level. What happened a lot, though, was the “noisy neighbor” problem: servers on the same network segment that were either origin or target of a DDoS attack which then impacted the whole segment. The only way to solve these problems back then was to notify datacenter staff who then identified and isolated the server involved in the attack; a procedure that usually takes 5 to 10 minutes. We realised that the distribution of our servers exposed our infrastructure too much to these problems and decided to move into dedicated racks that housed only our own servers. With that change, things got much quieter. This is a big advantage our bare-metal infrastructure has over cloud-based solutions: we have full control over where our IT resources are located and how much or how little they share with others. In November, our datacenter provider experienced a massive DDoS attack with no clear target and a traffic volume that caused problems even at their uplink carrier. It was a rough weekend for us and our customers. Interestingly, after that event, we’ve not experienced any other serious network problem at all. Looks like our datacenter provider made some effective changes. Additionally, they officially announced plans to overhaul their network infrastructure and to put anti-DDoS systems in place.

Results

We’re proud to say that we achieved quite a bit in 2013, especially with freistilbox. To be honest, we had started the year with a massive low because we launched the platform before it was ready for prime time. The negative feedback from our customers made it clear that by rushing the launch, we had caused big disappointment and lost a lot of our customer’s trust. Some customers also questioned the decision to divert manpower from the production platform to freistilbox Solo. In consequence, making the system reliable and efficient was our topmost priority during the year and we put a lot of hard work and many hours into it. I’m very happy to say that we now managed to give freistilbox the high level of quality it should have had from the start. In a round of customer calls I did in December, we got a lot of praise for how well the platform is working and how good a job we do with improving and supporting it. We are incredibly grateful to our customers for all their feedback.

We received 1,693 support requests last year, down 28% from 2,367 in 2012. The first quarter had the most new tickets (479) and Q4 ranked lowest (367). This is a good sign that we’re making progress in improving both the quality and the ease of use of freistilbox.

For a team with only two DevOps engineers, we’ve certainly created a great product for hosting Drupal and WordPress sites. And we’re only able to keep up doing the daily business as well as the tech support and 24/7 on-call because we’ve learned to collaborate effectively. The key to effective teamwork is communication, especially in a distributed team. Judging from the fact that we replaced Campfire as our communication backbone with HipChat, and heavily use Confluence for internal documentation, JIRA for task and project management and BitBucket as our code repository, Atlassian products seem to fit our needs quite well. Another communication product that I first didn’t expect to catch on actually became one of our most important channels: Sqwiggle. It took a bit getting used to a slight “Big Brother” feeling but we highly appreciate being able to tell at a glance who’s currently at their desk and to start a video call with a single click.

Over the recent months, we also reshuffled our areas of responsibility in order to put our strengths to use more effectively. Markus took over some of my technical tasks and now takes care of daily operations. I’ve assumed a more strategic position in business development.

Outlook

Laying stronger foundations: We’ve chosen Ireland as our new base of operations because of its growing importance as a tech hub for Europe. Our new company “freistil IT Ltd” will soon take over all business we’ve done so far as the partnership “freistil IT GbR”. The first benefit this change brings for our existing customers is that we’ll now be able to accept credit card payments.

Growing our team: We’re great in automating IT processes but we can’t automate innovation, that needs pure brain power. And for better “load distribution”, we need more “nodes”. We’ve been looking for quite some time and finally found two talented system administrators that are going to join our team over the coming weeks. This means we need to learn quickly about hiring and being a great employer.

Expanding our business: So far, we’ve mostly gained new customers by word of mouth. We love the fact — and we can’t be thankful enough for it — that our customers are so happy with freistilbox that they recommend our hosting platform to their friends and clients. To really expand our customer base internationally, though, we need to increase our sales and marketing activities. Since neither Markus nor I have a strong background in these areas, we decided to get external help. Thanks to an Enterprise Ireland Mentor Grant, a consultant experienced in international business will work with us on our growth strategy over the coming months.

It’s obvious that we’ve learned a lot of things in 2013, sometimes the hard way. We’re thankful for all the good will, feedback, advice and encouragement we’ve received from our families, friends and our customers. Every day, we’re getting better in fulfilling our mission: Making sure that our customers can work efficiently and sleep peacefully. To a great 2014!

---

[1] I recently found out that we were destined to go to this conference: Who would have thought that “Markus & Jochen” is an anagram for “Shamrock & June”!

• Read more

The important aspects for working in a remote team like here at freistil IT are very well explained in Mark Campbell’s blog entry “How to work remotely as a software developer”:

• Workspace
• Routine
• Setting limits
• Time boxing
• Communication

I’ve found the communication aspect the most essential one, because a remote team needs to compensate for the distance between coworkers. In an office, there are countless opportunities to have short exchanges to bounce ideas off each other, discuss findings and talk about other things. So, to keep round-trip times short in a virtual office, it’s important to use fast communication channels like HipChat or Sqwiggle. (We’ll expand on how we actually use these tools in later posts.)

There’s another great insight in Mark’s post I can wholeheartedly agree to:

“Working from a coffee shop is great. People will ignore you, you get hopped up on caffeine, and there’s a constant noise level about you.”

Oh, and by the way, Mark’s points are fully valid also for system administrators.

(Picture credit: Tracy Ruggles)

• Read more