IT

Vilnius kindergarten registartion – complete failure

Aleksandr Stankevic, , 5 Comments

Finally came the day when I also have to submit a request for my kid to get a place in a kindergarten!

On 25th of March (on my son’s birthday), at 8 am, the registration for kindergarten should have started on vilnius.lt website. After waking up, the first thing i did was try to login. At 7:55 am, the website opened. “Hooray!” – I thought – “it didn’t crash this time!”

5 minutes later i understood, that nothing has changed…

 

vilnius.lt 504

Some of my thought about this, while i tried to sign up my kid for 5 hours….

This system has been working for a few years now, and every year there’s the same issue. They tried to get the registration system working this year in January, but the system couldn’t handle all the traffic. The registration date was changed to March 25th, with a few months of time to get the system fixed and optimised. Unfortunately, vilnius.lt IT department had failed to do this job. It was said that everything got fixed, but probably nobody did any testing or benchmarking to see if it really got fixed.

Some thoughts from my side on what could have been done better:

No CDN?

All the media files (images/css/js) are served from the same http://www.vilnius.lt website, without using any CDN. If a CDN was used or they at least had separate servers for serving static content, the situation would improve a lot, as the main http://www.vilnius.lt server wouldn’t be overloaded with these requests during  the peak time.

There are free CDN’s, paid CDN, or you could turn on the CDN only during this event which generally takes place only a few days during the year.

AJAX is cool!

The kindergarten registration system uses a lot of ajax requests. But … are all of them really needed? For example – you are required to enter your personal identity code during the registration. When you fill it out, there’s an ajax request made to validate the personal identity number ( there’s an algorithm for that) , then there’s another request to get the birthdate out of the PIN  (it’s part of the PIN, digits 2-7 ), then there’s another request for the same ( not sure why … ). This part can be easily done on JS side, as the algorith to validate this information isn’t very hard. And then make the server side validation during submit. But no, it’s better to make 3+ requests to overloaded servers just for this thing.

Lets not forget about all other fields! I.e. address. If I live on a street ‘Zaumenhofo’, and i start typing in ‘Zaumenhof’, then it sends an ajax request for every letter after the third one to get the list of streets based on the input field to try to autocomplete the street name. That is – ‘zau’, ‘zaum’, ‘zaume’, ‘zaumen’, ‘zaumenh’, ‘zaumenho’, ‘zaumehof’, ‘zaumenhofo’ – that’s 8 requests! Of course, then the system works properly, it should be enough to type in ‘zau’ and the system will give you a list of streets. But when the servers are overloaded, you’re not even getting the replies to the ajax requests, and keep on writing the street name – which generates even more requests. Also, you need to write the details for mom, dad and kid separately, which means for ‘zaumenhofo’ street, that’s 8 requests (for each letter ) * 3 = 24 additional requests.

Every other field works in the same way – an ajax request is sent for each dropdown or input field. There’s probably a couple hundred of them for the whole registration process – but i didn’t count.

You could just send these to separate servers which only validate this information so that you don’t overload the main servers that do eveeeerything else.

The winter came early this year!

Like every year so far, the winter came unexpectedly! Mostly during the second half of December, and the road cleaning guys are always not ready for it! Who could think of winter actually coming, in December.

We have the same situtaion with vilnius.lt kindergarten. HOW could they know that the registration will start on March 25th. IF they knew it would, they would get some additional virtual machines running to cope up with the load, RIGHT?

No…

In other words – they knew the date, they knew that the systems are always overloaded, but why can’t they just get ready for that? I understand that buying new servers just for a day or two is not very cost effective. But we live in 2015! There’s a lot of *be ready, a buzzword will be used in a minute* CLOUDS, where you can buy a server, or two, or even 100 for only a few days. And they are not expensive! You could buy 20 virtual machines in amazon, connect them to your app/backend, and pay something like 20-50 $ for those few days. The price is not big, but you wouldn’t get so shamed in front of whole country and world!

Maybe lets start sending emails with pidgins?

Not related to performance, but it’s hard not to see the note about trying to look in your spam folder if you haven’t received an email soon.

Lets check the inbox:
Return-Path: NO-REPLY@vilnius.lt
Received: from mail.vilnius.lt (mail.vilnius.lt. [195.182.82.88])
by mx.google.com with ESMTPS id ol7si1269421pbb.71.2015.03.25.01.57.07
for XXX
(version=TLSv1 cipher=ECDHE-RSA-AES128-SHA bits=128/128);
Wed, 25 Mar 2015 01:57:08 -0700 (PDT)
Received-SPF: none (google.com: NO-REPLY@vilnius.lt does not designate permitted sender hosts) client-ip=195.182.82.88;
Authentication-Results: mx.google.com;
spf=none (google.com: NO-REPLY@vilnius.lt does not designate permitted sender hosts) smtp.mail=NO-REPLY@vilnius.lt
Received: from HUB2.vilnius.vilnius.lt (10.13.3.4) by mail.vilnius.lt
(192.168.2.2) with Microsoft SMTP Server (TLS) id 14.2.347.0; Wed, 25 Mar
2015 10:53:24 +0200
Received: from w3r4.vilnius.lt (10.187.87.164) by HUB2.vilnius.vilnius.lt
(10.13.3.224) with Microsoft SMTP Server id 14.2.347.0; Wed, 25 Mar 2015
10:55:03 +0200
Date: Wed, 25 Mar 2015 10:55:57 +0200
To: XXX
From: no-reply@vilnius.lt
Subject: Prisijungimo duomenys
Message-ID: <e3cf4223db7449aa8e31488bfd238c9b@w3r4.vilnius.lt>
X-Priority: 3
X-Mailer: PHPMailer 5.2.9 (https://github.com/PHPMailer/PHPMailer/)
MIME-Version: 1.0
Content-Type: multipart/alternative;
boundary="b1_e3cf4223db7449aa8e31488bfd238c9b"
Return-Path: no-reply@vilnius.lt
X-Originating-IP: [10.187.87.164]

First thing we notice is the strange email forwarding between servers (from server A 10.187.87.164, to server B 10.13.3.224 (and 192.168.2.2), to server C 10.3.3.4). Then we can see that SPF is not used  (it says which IP can send emails from @vilnius.lt), nor DKIM (says that an email sent from @vilnius.lt is actually sent from that server), nord DMARC (which says what should be done with emails that are not DKIM/SPF valid). Maybe it’s time to start using an 10 year old technology (except DMARC, which is newer than 10 years) in the government sector aswell? These technologies are VERY expensive (Free), are VERY hard to set up (probably takes an hour to setup without prior knowledge about them) and need a lot of specialists time.

A slowdown in data processing, or an completely not working service?

Everyone on the radio keeps saying that the kindergarten registration works, but has a slowdown in data processing. That’s a very strange phrase. For me, “slowdown in data processing” is when the data is submitted without any problems, but it takes a longer time to process it once it’s submitted. Unfortunately, in this case, the situation is very different. Now, the lucky ones who are able to login, are able to submit the data. The unlucky ones can’t even log in, and can’t submit the data. The first ones are in one queue, the other ones are not in any different queue.

In example: if nginx is used, it has an option to always send users to the same backends based on a hash (i don’t know if that’s how it works on vilnius.lt, but this is just an example that “slowdown” != “not working). So, lets say we have 100 users with different IP’s and 3 servers. Then, 33% of users (A) go to server 1, 33% of users (B) go to server 2, and the other 33% of users (C) go to server 3. All is good, right?

No.

If 100% A users and 100% B users go to registration page – the registration won’t even open for them, because server 1 and server 2 will be overloaded. And if only 20% of users C go to registration, then they’re first in the queue (the server 3 won’t be as overloaded as  server 1 and 2 ).

That’s why i think that it’s not a “slowdown” but rather a completely not working service, and it’s a matter of luck/algorithm wherever you’ll be sent to the server which is not overloaded.

EDIT: this post was translated to english, as my Lithuanian grammar is very poor and there were lots of negative comments about this.

Vilnius kindergarten registartion – complete failure

Print your … spare server part!

Aleksandr Stankevic, , , , Leave a comment

I’ve got some pretty old “server” here and it’s disk said goodbye to me a few weeks/months ago.

I finally found some time to clean it up – clean all the dust inside, apply some toothpaste thermal paste to the heatsink and replace the disk. Unfortunately i didn’t take a picture of how awful it looked before, but here’s a cleaned up version.

cleaned up server
cleaned up server

As you can see – it has a bracket for one disk, and there’s space for another one. It worked with one disk so far, but it would be a waste of space not to add the second one – the disks are so cheap now. I tried to search for the other bracket but couldn’t find it anywhere, and i couldn’t find a spare one to buy at the local stores.

We’ve got a MakerBot in the office out here. So, i’ve asked a friend if he can try printing one for me. The next day he comes back with this:

original drive bracket
original bracket
3d printed disk bracket
3d printed bracket
brackets with drives
both brackets with drives

I was a bit skeptic about the real use of 3d printers in homes / offices, as the printed examples i usually see are keychains or other useless stuff, but now i might change my mind. maybe.

P.S. Not sure how will the heat influence the bracket – hope i won’t see any smoke coming out of the case…

Print your … spare server part!

Annoying Notes app formatting

Aleksandr Stankevic, , , , 1 Comment

As a Mac user, i have been (ab)using the Notes app quite a lot – i always copy commands/output/text from terminal or web pages, and it always annoyed me that Notes app, by default, would always format the text – create links, change the quotes, try to have text background (i.e. black when copying from Terminal). That’s the most annoying thing ever…

I remember i tried to look in settings to see if there’s a way to disable this, but I didn’t find anything when i looked. Today, it annoyed me too much again, so i thought i need to find a different software for that… but first let me try to look at it again.

So, after some googling around, i found a solution – you can remap the ⌘V key to paste without formatting!

Steps to achieve that:
– Go to System Preferences -> Keyboard -> Shortcuts -> App Shortcuts
– Add a new shortcut
– Choose “Notes” as the aplication
– Name the Menu Title “Paste and Match Style”
– In Keyboard Shortcut press ⌘V

This should re-map the ⌘V to use ‘Paste and Match style’ rather than the default Paste method.

Life just got a bit less annoying 🙂

Annoying Notes app formatting

WordCamp Russia 2013

Aleksandr Stankevic, , , , , , , 1 Comment

A few months ago, Konstantin Kovshenin pinged me and asked if I want to come over to Moscow for a WordCamp. Without thinking too much, i have agreed. Unfortunately, that had some consequences – i had to give a speech in a WordCamp.

Why unfortunately? Because:

  • I had never attended a WordCamp – and having my first WordCamp as a speaker was very scary
  • The presentation has to be in Russian. I do speak Russian, but I’ve never learned it anywhere, only from friends / TV – so lots of errors expected
  • I had never given a presentation to a big group of people who cared about what i say – usually people just ignore me 🙂

Well, but i still agreed on it!

As always, i had delayed to create the presentation up to the last week. If not for the few  pings from Konstantin, I’d probably delay it until the last minute.

I’ve got to Moscow on Friday morning, and spent most of the day working from the hotel and Starbucks. Then, in the evening, we had a pre-WordCamp meetup with all the volunteers / speakers. The meetup was in some lovely caffe called ‘Chocolate‘ and you could see a part of the Red Square from it. I’ve met all those nice and smart people! Everyone looked so brave, but i knew that deep inside they were also as afraid of the next day as i was 🙂

Next day i’ve got to the Digital October lounge around 8am to help out with anything that’s needed, but it looked like almost everything was planned and ready. People started gathering 15 minutes before the registration, which was at 9am. There was around 150 people in total, which isn’t that much for Moscow, but it’s quite a lot for a first WordCamp.

Konstantin was leading the first part of the WordCamp, and Nikolay Mironov was leading the second part. They had some nice idea of presenting the speakers in some funny way – although the funny part was usually truth.

First speaker, the bravest woman of them all, was Olga (Ольга Горяева) with her presentation about using WordPress for everything else rather than just a blog – news sites, portfolio websites, internet shops, radio/music streaming websites, twitter analytics and etc. Of course i knew this, but it was a really good presentation for all those users who still think that WordPress is only for blogs. Double points for being the first speaker!

Second speaker – Nikolay Mironov – was talking about portfolio websites and freelancing. Lots of freelancers don’t have a portfolio website, and it’s not that hard to make one (pretty easy actually). Also, he gave lots of hints on how to be a good freelancer, where to search for clients, how to make the clients stay and etc. The way he did the speech was very attractive – he was very dynamical / active.

I missed the third speech as i was getting caffeinated and nicotinated before the next speaker … me! But from the number of questions after the presentation i understand that it was very interesting, especially to some people interested in the topic.

So, my presentation probably didn’t go very well, but well, there’s always a first time for everything, and it’s usually not ideal on the first time 🙂

My presentation took only about 20 minutes (whereas usually other speakers spoke for about half an hour) and then i had a number of questions. I was very nervous so i probably gave some false answers to one or two questions (i.e. the hdd brand we use the most). Also, some of my answers were short and i could have given more information about the answer.

My presentation was the last one before the lunch break. We had some subway sandwiches ready for us (separate sandwiches for the vegetarians). I was very hungry, nom nom 🙂

Next presentation was from Sergey Biryukov about taking part in WordPress development. It was nice to hear how WordPress development works, the meetings for developers, where to start, where each part of development happens and etc.

The next one was Konstantin and he gave a presentation about increasing the performance of your WordPress website. It was probably the most interesting presentation for me, as a sysadmin. He told about some interesting situation where he had to create some custom tables with MySQL on insert/update triggers to better use the indexes on the fields he needed. If i did this presentation, i would probably have missed some parts he talked about.

Unfortunately, i had to grab my notebook at this time and missed out on the last two presentations made by Gennadiy Kovshenin – Unit testing in WordPress, and Anna Ladoshkina – Adaptive content in WordPress, but i will  definitely watch these two presentations when they’ll come out on WordCamp.tv

After that, we had some after party in the local bar called “Progress Bar”. For some reason, most of the people went home, and didn’t want to spend their Saturday evening having a chat with geeks. By the way, if you ever go to Progress Bar – don’t forget to look down when you go to the bathroom, it has some interesting view out there 🙂

Progress Bar floor

Overall, the WordCamp was great, i met a lot of interesting people, the presentations were also interesting. If you’ve got a WordCamp anywhere around your city – please go, you’ll really like it!

WordCamp Russia 2013

Two new softwares – 1Password and Spotify

Aleksandr Stankevic, , Leave a comment

Recently I started to use two new softwares: 1Password and Spotify

1Password is a password storing software from AgileBits. I usually generate a random password for each service/website and store them encrypted on my own server. I have done this because i had to have access to it from different places (notebook, pc, work notebook). Last time i tried to find a similar software i couldn’t find one which would let me access it from different computers.

It has plugins for Firefox/Safari/Chrome which lets me access the passwords quickly, a ‘copy’ button to quickly copy the password (no more marking and cmd+c!), a password generator and a search tool. It also can store the password safely ‘in the cloud’.

I’m trying to move all my passwords to 1Password now while changing them during the move, but i’ve got really a lot of them so it takes time!

Second software is Spotify. I have heard of it many times before, but didn’t try it as Spotify isn’t friendly with Lithuania 😦 I’ve used a US proxy to register with them and boom, i’m using it now! I hope i find a way to get a premium account when my free trial finishes 🙂

Spotify is a music service which gives access to millions of songs. I like that i can listen almost to any song in a good quality, and legally. I could find almost all songs i like, except for russian songs. It has some of them, but definetly not enough.

Two new softwares – 1Password and Spotify

issues with my new Air “13

Aleksandr StankevicLeave a comment

I’ve got a new notebook lately – a MacBook Air 13″. But it is really acting strangely 😦

I keep getting the magic beach ball all the time, apps are hanging, the FireVault encryption finished with a MacOS crash.

For example, i run the ‘ping’ against a host, and it hangs. It doesn’t print anything for a long time as if it was trying to resolve a host. But if i try to run a ‘dig’ in another window – it returns the information instantly. If i run ‘host’ – it also hangs.

Might be something related to a Network or DNS resolution. I tried to have  a clean install of my Mac to see if it helps.

I’m waiting for the next crash/hang to try to debug the issue.

 

issues with my new Air “13