• Help grow the Triggit ad serving platform which currently handles hundreds of thousands of requests per seconds. Almost all requests require complex decision making with large sets of sparse data and must be returned within tens of milliseconds
• Work on our next generation infrastructure components in Golang
• Knowledge of a modern scripting language like Python or Ruby
• Work with infrastructure components (ZeroMQ, Postgres, Cassandra)
• Contribute business logic to big data projects
• Work with a dynamic team on a variety of projects and new product features

Requirements:

• Experience programming in Golang either for fun or through producing production code.
• Must be open to learning new
  programming languages (polyglot
  programmer or desire to become one)
• 3-5 professional work experience
• Elegant programming style and ability to be an individual contributor
• Very solid understanding of RDBMS (we use Postgres), SQL, and database normalization.
• Strong Unix fundamentals and thorough understanding of HTTP and all its intricacies.
• A detail-oriented approach to development and an interest in developing new skills
• Able to work in our San Francisco HQ

Either apply here or feel free to email Joanne(at)triggit.com with your resume and any past projects/open source stuff that you feel is a good representation of your work.

Get information on how to apply for this position.

As a co-inventor of Erlang, Robert Virding is probably one of the best persons in the world to teach you all you need to know about Behaviours. It’s a bit like getting piano lessons from Bach.

So we’re extremly thrilled to announce that this year Robert Virding will be teaching this year’s OTP Express course starting on 10 June. What will you learn?

• Use existing Design Patterns supported by Erlang and OTP.

• Apply Generic Behaviours, Finite State Machines and Event handler Patterns.

• Use the Supervisor and Application Behaviours Patterns.

• Write your own Design Patterns.

• Structure large Erlang based systems.

To register for the course, go here

As a Test Engineer you are part of an agile development team that consists of around 5-8 members. You work closely to the developers and will develop the ways you work with testing within your team. The developers do unit tests and you will mostly focus on white-box testing but also some black-box testing. The test Engineer also creates tools and test frameworks when needed to be used inside the team.

We believe that you should automate what can be automated. This means that your mission is to implement test cases, automate tests and make sure we deliver in accordance to our high demands. Since we are forming our future test organisation it is a great opportunity to help setting up and develop our ways of testing. Make your ideas be part of our overall plan.

Requirements:

• Degree in Computer science or similar degree with focus on programming
• 3+ Years of experience in testing in medium- or largescale software development
• Experience of agile software development (TDD / Scrum / Kanban)
• Broad skills in programming (preferably functional programming such as Erlang, Lisp, Scala, Scheme, F#, Haskell etc)
• Great skills in automation and scripting (Perl, Bash, Python etc)
• Broad experience in white-box testing frameworks such as xUnit

Preferred skills:

• ISTQB-certificate (or ISEB, CSTE)
• Experience of BDD, Behaviour-driven Development, or ATDD, Acceptance Test Driven Development
• Experience in different higher level testing frameworks (Selenium, Cucumber etc)

We offer you an international working environment filled with smart and ambitious colleagues. As an employee at one of Sweden’s fastest growing companies, you will play an important role in taking Klarna to the next level. You won’t work for Klarna, you’ll work with Klarna.‬

For questions please contact Philip Andrén at philip [dot] andren [at] klarna [dot] com or +46 (0) 76-526 00 70. We recommend you to apply as soon as possible, selection and interviews are held continuously. 

Location Stockholm, Sweden

Get information on how to apply for this position.

Your job as a senior developer is to take part in the entire development process, from gathering requirements to delivery. The challenge is to provide solutions that scale. The team members have a high technical level and are expected to work independently within the team to solve complex problems, from analyzing the problem to designing, planning and implementing the solutions.

We strive to always select an appropriate tool for solving the particular problem at hand. In order to achieve this we aim to have a diverse palette of skills in every team. Thus, it is not vital that you have previous experience with the specific tools that we are already using, because in our eyes, a great developer will always be a great developer.

Required Qualifications

• 10 years of development experience with a proven track record of high quality software development
• MSc in Computer Science or equivalent
• Experience from agile methodologies (Scrum, Kanban etc)
• Knowledge in many different domains and programming paradigms
• Deep understanding of algorithms, data structures and complexity up to and beyond programming language implementation and semantics
• Value and know the power of testing and TDD

You…

• value learning and solving problems together with your team mates.
• know the power of automation and avoid repeated manual work.
• know the value of and practice reviews relentlessly.
• write code only when necessary.
• have a rubber duck.

If you have any question please email or call Philip Andrén at: philip [dot] andren [at] klarna [dot] com or +46 765 26 00 70.

Location Stockholm, Sweden

Get information on how to apply for this position.

"April 2013" could not be found.

There are a couple of options to fix this. Some form of escape character is one, so an embedded quote is preceded by, say, a backslash. That works, but now you’ve got to explain a second concept in order to explain strings. Another option is to allow both single and double quotes. If your string contains single quotes, enclose it in double quotes, and vice-versa. A hand goes up, and someone asks about how to enter this string:

"April 2013" can't be found.

Ugh. Now you have two kinds of string delimiters, and you still need escapes. You need to explain these special cases up front, because they’re so easy to hit.




What if instead falling back on the unwritten rule of using single and double quotes, strings were demarcated by something less traditional? Something that’s not common in Latin-derived languages? I’ll suggest a vertical bar:

|"April 2013" can't be found.|

That may be uncomfortable at first glance, but give it a moment. Sure, a vertical bar will end up in a string at some point—regular expressions with alternation come to mind—but the exceptional cases are no longer blatant and nagging, and you could get through a beginning class without even mentioning them.




(If you liked this, you might enjoy Explaining Functional Programming to Eight-Year-Olds.)

If you’re looking for some code that implements what that post describes, look no further than DeadZen‘s etsgive repository. DeadZen is a very experienced Erlang developer, so it’s no surprise that his example code is straightforward and clean.

Just as described in my post, DeadZen’s supervisor code starts two workers, one that manages the table and one that uses it. The table manager process takes the following steps:

1. traps exits
2. links to the table user worker process (the supervisor starts the table user worker first)
3. creates the ets table
4. sets itself as the table heir process
5. gives ownership of the table away to the table user worker process

The table user worker process first handles the transfer message resulting from the manager giving it ownership of the table. Once it becomes the owner, it can handle a call to increment a counter in the table and a call to check the table contents. It can also handle a call to die, which you would issue interactively from an Erlang shell to exercise the code. This causes the worker process to die, which means the table manager regains ownership of the table because it’s the table’s heir. The table manager then waits for the supervisor to start a a new table user worker process, and once it’s up and running, the manager links to it and then transfers table ownership to it.

This written description might be difficult to work through, but fear not, DeadZen also supplies an example Erlang shell session showing how it all works. Clone his repository and try it out for yourself!

Dynamo based systems work too hard. SimpleDB/DynamoDB, Riak, Cassandra and Voldemort are all based, at least in part, on the design first described publicly in the Amazon Dynamo Paper. It has some very interesting concepts, but ultimately fails to provide a good balance of reliability, performance and cost. It’s pretty neat in that each transaction allows you dial in the levels of redundancy and consistency to trade off performance and efficiency. It can be pretty fast and efficient if you don’t need any consistency, but ultimately the more consistency you want the more have to pay for it via a lot of extra work.

Network Partitions are Rare, Server Failures are Not

... it is well known that when dealing with the possibility of network failures, strong consistency and high data availability cannot be achieved simultaneously. As such systems and applications need to be aware which properties can be achieved under which conditions.

For systems prone to server and network failures, availability can be increased by using optimistic replication techniques, where changes are allowed to propagate to replicas in the background, and concurrent, disconnected work is tolerated. The challenge with this approach is that it can lead to conflicting changes which must be detected and resolved. This process of conflict resolution introduces two problems: when to resolve them and who resolves them. Dynamo is designed to be an eventually consistent data store; that is all updates reach all replicas eventually.

- Amazon Dynamo Paper

The Dynamo system is a design that treats the probability of a network switch failure as having the same probability of machine failure, and pays the cost with every single read. This is madness. Expensive madness.

Within a datacenter, the Mean Time To Failure (MTTF) for a network switch is one to two orders of magnitude higher than servers, depending on the quality of the switch. This is according to data from Google about datacenter server failures, and the publish numbers of the MTBF of Cisco switches (There is a subtle difference between MTBF and MTTF, but for our purposes we can treat them the same)

It is claimed that when W + R > N you can get consistency. But it’s not true, because without distributed ACID transactions, it’s never possible to achieve W > 1 atomically.

Consider W=3, R=1 and N=3. If a network failure or more likely a client/app tier failure (hardware, OS or process crash) happens during the writing of data, it’s possible for only replica A to receive the write, with a lag until the cluster notices and syncs up. Then another client with R = 1 can do two consecutive reads, getting newer data first from a node A, and older data next from node B for the same key. But you don’t even need a failure or crash, once the first write occurs there is always a lag for the next server(s) to receive the write. It’s possible for a fast client to do the same read 2 times again, getting a newer version from one server, then an older version from another.

What is true is that if R > N / 2, then you get consistency where it’s not possible to read in a newer value, then a subsequent read get’s an older value.

For the vast majority of applications, it’s okay for a failure leading to temporary unavailability. Amazon believes it’s shopping cart is so important to capture writes it’s worth the cost of quorum reads, or inconsistency. Perhaps. But the problems and costs multiply. If you are doing extra reads to achieve high consistency, then you are putting extra load on each machine, requiring extra server hardware and extra networking infrastructure to provide the same baseline performance. All of this can increase the frequency of a component failure and increases operational costs (hardware, power, rack space and the personnel to maintain it all).

A Better Way?

What if a document had 1 master and N replicas to write to, but only a single master to read from? Clients know based on the document key and topology map which machine serves as the master. That would make the reads far cheaper and faster. All reads and writes for a document go to the same master, with writes replicated to replicas (which also serve as masters for other documents, each machine is both a master and replica).

But, you might ask, how do I achieve strong consistency if the master goes down or becomes unresponsive?

If when that happens, the cluster also notices the machine is unresponsive or too slow and removes it out of the cluster and fails over to a new master. Then the client tries again and has a successful read.

But, you might ask, what if the client asks the wrong server for a read?

If all machines in the cluster know their role and only one machine in the cluster can be a document master at any time, and the cluster manager (a regular server node elected by Paxos consensus) makes sure to remove the old master, and then assign the new master, and then tell the client about the new topology. Then the client updates it’s topology map, and retries at the new master.

But, you might ask, what if the topology has changed again, and the client again asks to read from the wrong server?

Then this wrong server will let the client know. The client will reload the topology maps, and re-request from the right server. If the right master server isn’t really right any more because of another topology change, it will reload and retry again. It will do this as many times as necessary, but typically it happens only once.

But, you might ask, what if there is a network partition, and the client is on the wrong (minor) side of the partition, and reads from a master server that doesn’t know it’s not a master server anymore?

Then it gets a stale read. But only for a little while, until the server itself realizes it’s no longer in heartbeat contact with the majority of the cluster. And partitions like this are the among the rarest form a cluster failure. It will require a network failure, and for the client to be on the wrong side of the partition.

But, you might ask, what if there is a network partition, and the client is on the wrong (smaller) side of the partition, and WRITES to a server that doesn’t know it’s not a master server anymore?

Then the write is lost. But if the client wanted true multi-node durability, then the write wouldn’t have succeeded (the client would timeout waiting for replicas(s) to receive the update) and the client wouldn’t unknowingly lose data.

What I’m describing is the Couchbase clustering system.

Let’s Run Some Numbers

Given the MTTF of a server, how much hardware and how quickly must the cluster failover to a new master and still meet our SLAs requirements vs a Dynamo based system?

Let’s start with some assumptions:

We want to achieve 4000 transactions/sec with 3 node replication factor. Our load mix is 75% reads/25% writes.

We want to have some consistency, so that we don’t read newer values, then older values, so for Dynamo:

    R = 2, W = 2, N = 3

But for Couchbase:

    R = 1, W = 1, N = 3

This means for a Dynamo style cluster, the load will be:
Read transactions/sec: 9000 reads (reads spread over 3 nodes)
Write transactions/sec: 3000 writes (writes spread over 3 nodes)

This means for a Couchbase style cluster, the load will be:
Read transactions/sec: 3000 reads (each document read only on 1 master node, but all document masters evenly spread across 3 nodes)
Write transaction/sec: 3000 writes (writes spread over 3 nodes)

Let’s assume both systems are equally as reliable at the machine level. Google’s research indicates in their datacenter each server has a MTTF of 3141 hrs or 2.7 failures per year. Google also reports a rack failure (usually power supply) of 10.2 years, roughly 30x a reliable as a server, so we’ll ignore that to make the analysis simpler. (This is from Googles paper studying server failures here)

The MTBF of Cisco network switch is published at 54,229 hrs on the low end, to 1,023,027 hrs on the high end. For our purposes, we’ll ignore switch failures, since the failures affects availability and consistency of both system about the same, and it’s 1 to 2 orders of magnitude rarer than a server failure. (This is from a Cisco product spreadsheet here)

Assume we want to meet a latency SLA 99.9% of the time (the actual latency SLA threshold number doesn’t matter here).

On Dynamo, that means each node can fail SLA the 1.837% of the time. Because it queries 3 nodes, but only uses the values from the first 2 nodes and the chances of SLA failure are the same across nodes, the formula is different (only two must meet the SLA):

    0.0001 = (3 − 2 * P) * P ^ 2

or:

    P = 0.001837

On Couchbase, if a master node fails, it must recognize it and fail it out. Given Google’s MTTF failure above and it can fail out a node in 30 secs, and let’s say it will take 4.5 minutes for it warm up the RAM cache, given 2.7 failures/year with 5 minutes of downtime for each before a failover completes, then queries will fail 0.00095% of time due to node failure.

For Couchbase meet the same SLA:

    0.0001 = P(SlaFail) + P(NodeFail) - (P(SlaFail) * P(NodeFail))

    0.0001 = P(SlaFail) + 0.0000095 - (P(SlaFail) * 0.0000095)

    0.0001 ~= 0.00009 + 0.0000095 - (0.00009 * 0.0000095)

Note: Some things I’m omitting from the analysis are when a Dynamo node fails the lower latency requirement from meeting the SLA for 2 nodes vs. 3 (it would drop from 1.837% to ~0.05%), and also the increased work on the remaining servers when a Couchbase server fails. Both are only temporary and go away when a new server is added back and initialized in the cluster, and shouldn’t change the numbers significantly. Also there is the time to add in a new node and rebalance load on it. At Couchbase we work very hard to make that as fast and efficient as possible. I’ll assume Dynamo systems do the same, that the cost is the same and omit it, though I think we are the leaders in rebalance performance.

With this analysis, a Couchbase node can only fail it’s SLA 0.9% of the requests, and a Dynamo node can fail it 1.837%. Sounds good for Dynamo, but it must do for 2X the throughput per node on 3x the data, and with 2x the total network traffic. And for very low latency response times (our customers often want sub-millisecond latency) typically meeting the SLA means a DBMS must a large amount of relevant data and metadata in RAM, because there is a huge cost for random disk fetches on latency. With disk fetches 2 orders of magnitude slower on SSDs (100x), and 4 orders of magnitude slower on HDDs (10000x) the disk accesses pile up faster without much more RAM, so do the latencies.

So each Dynamo node can fail it’s SLA at a higher rate is very small win when it will still need to keep nearly 3X the working set ready in memory because each node will be serving 3x the data at all times for read requests (it can fail it’s SLA slightly more often, so it’s actually about 2.97x the necessary RAM), and will use 2x the network capacity.

Damn Dynamo, you sure do work hard!

Now Couchbase isn’t perfect either, far from it. Follow me on twitter @damienkatz. I’ll be posting more about the Couchbase shortcomings and capabilities, and technical roadmap soon.

Headquartered in Dublin and Berlin, SumUp is a dynamic and innovative start-up company developing cutting-edge mobile payment solutions. With a global workforce of over 130 young, energetic and dedicated staff, our product is set to revolutionalise the world of payment services.

The company offers an application and card reader that can turn any tablet or smart phone into a credit card terminal, which gives small merchants the ability to accept card payments as well as replace their expensive cash registers. SumUp enjoys impressive global reach that is ever expanding. We are currently operating in Ireland, UK, Germany, Austria, Italy, Spain, the Netherlands, France, Belgium and Portugal.

We are looking for vibrant, practical and resourceful future employees to join our great company.

The Role

• Develop quality solutions and features for the SumUp product in a timely manner.
• Support SumUp production services and components.
• Collaborate with team members in highly agile scrum teams to design, develop, test and deploy software projects.
• Work to troubleshoot, analyse and code fixes for reported software issues.
• Plan and size coding, testing and deployment tasks.
• Bring an open-minded opinion about technology and processes to all technological discussions within the team; and be flexible in all team dealings.
• Communicate and collaborate with colleagues outside of the Software Department to deliver value for Sumup.
• Focus at all times on delivering the best user experience possible to SumUp customers and internal users.
• Represent SumUp in a professional manner at external software community meetups.

IT Skill Set Required

Essential

• 5+ years programming experience in a product development environment.
• Practical Erlang development experience & good working knowledge Erlang-based libraries.
• Experience in developing fault-tolerant distributed systems.
• Experience with scaling systems to large numbers of concurrent users.
• Flexibility and open-mindedness in using tools, technologies and new ideas.
• Strong communication skills; capable of working closely and smoothly with team members to deliver business goals. Team cooperation is essential.
• Commonsense and pragmatism in developing new features with a strong focus on delivering business value.

Desirable

• Knowledge of other programming languages (Ruby Python, Objective-C, C++, etc.) a plus.
• Familiarity with the mobile payments industry is desirable. Knowledge of EMV2 and PCI standards is a MAJOR plus.
• A good working knowledge of SQL - it doesn’t matter what flavour.
• Some experience with NoSQL-type systems (Riak, MongoDB, …)
• A good working knowledge of Linux/Unix (we use OS X for development)
• A broad knowledge and interest in other technologies (e.g. JSON, Web servers, networking, etc.)
• Experience in working in a team of developers in a flexible, agile environment.
• An interest in open-source technologies – willingness to collaborate in team hackathons.
• Experience in working in a startup environment would be an advantage.

General Skills profile

• Excellent communication skills: ability to communicate clearly and concisely, both verbally and in writing.
• Team working/interaction with others: strong team player and ability to interact with colleagues at all levels.
• Accuracy: methodical and detail conscious with high standards of accuracy.
• Strong analysis and problem solving ability: gather information and probe for facts. Weigh up options and produce practical and workable solutions within the overall defined architecture solution.
• Planning and organisation: strong time management and organisational skills, ability to prioritise own tasks.
• Good time management and ability to work to tight deadlines.
• Target Driven to remain focused on goals even when under pressure and maintain effective work despite challenges.
• Flexibility and openness to change: ability to react and adapt to changing priorities.
• Enthusiastic, dedicated and willing to learn.
• Conscientious with a “can do” attitude and proactive work ethic.

Get information on how to apply for this position.

AdRoll is an already profitable startup based in SOMA. We’re backed by some of the savviest investors in the space, including Peter Thiel, Accel Partners, and Foundation Capital.

We were just named the #1 fastest growing in CA by Inc.

Team up with a colorful, but tight-knit engineering squad of a dozen. Code alongside our group which includes the youngest engineer ever at Microsoft (our CEO), a former World Champion of Street Fighter, the president of Python Italy, and more!

We run the retargeting platform with the largest customer base and swim in the complex, yet exciting new world of real-time advertising.

Our technology is continually pushing the boundaries. We run a global infrastructure that handles millions of requests, participates in billions of real-time auctions, crunches big data in real-time while serving thousands of happy advertisers.

About You:

Do you dream of writing code that will be seen by nearly everyone you know? Do you yearn to build a 100% cloud-deployed infrastructure that deftly scales to billions of daily requests and is capable of self-healing when trouble strikes? Join us in making this vision a reality.

For this key position, you must write solid code, understand complex systems, competently evaluate scalability trade-offs (eg, disk vs cpu vs memory), and quickly troubleshoot issues. You learn fast and are excited to dig into the latest technologies. Our stack includes Python, Erlang, RabbitMQ, PostgreSQL, HBase, Hadoop, Redis and our own open-sourced solutions.

More than anything, you do your best work when you have lots of freedom and can take ownership of your projects from concept to finish.

Get information on how to apply for this position.

visitingTeam.outfield.center

The difference, though, is that programming isn’t done in real-time like scorekeeping. After the initial learning curve, 8 is much more concise, and the terseness is a virtue when recording plays with the ball moving between multiple players. Are we too quick to dismiss extremely terse syntax and justify long-winded notations because they’re easier for the uninitiated to read?




Suppose you have a file where each line starts with a number in parentheses, like “(124)”, and you want to replace that number with an asterisk. In the vim editor the keystrokes for this are “^cib*” followed by the escape key. “^” moves to the start of the line. The “c” means you’re going to change something, but what? The following “ib” means “inner block” or roughly “whatever is inside parentheses.” The asterisk fills in the new character.




Once you get over the dense notation, you may notice a significant win: this manipulation of text in vim can be described and shared with others using only five characters. There’s no “now press control+home” narrative.




The ultimate in terse programming languages is J. The boring old “*” symbol not only multiplies two numbers, but it pairwise multiplies two lists together (as if a map operation were built in) and also multiplies a scalar value with each element of a list, depending on the types of its operands.




That’s what happens with two operands anyway. Each verb (the J terminology for “operator”), also works in a unary fashion, much like the minus sign in C represents both subtraction and negation. When applied to a lone value “*” is the sign function, returning either -1, 0, or 1 if the operand is negative, zero, or positive.




So now each single-character verb has two meanings, but it goes further than that. To increase the number of symbolic verbs, each can have either a period or a colon as a second character, and then each of these have both one and two operand versions. “*:” squares a single parameter or returns the nand (“not and”) of two parameters. Then there’s the two operand version of “*.” which computes the least common multiple, and I’ll give it up now before everyone stops reading.




Here’s the reason for this madness: it allows a wide range of built-in verbs that never conflict with user-defined, alphanumeric identifiers. Without referencing a single library you’ve got access to prime number generation (“p:”), factorial (”!”), random numbers (”?”), and matrix inverse (”%.”).




Am I recommending that you switch to vim for text editing and J for coding? No. But when you see an expert working with those tools, getting results with fewer keystrokes than it would take to import a Python module, let alone the equivalent scripting, then…well, there’s something to the terseness that’s worth remembering. It’s too impressive to ignore simply because it doesn’t line up with the prevailing aesthetic for what constitutes readable code.




(If you liked this, you might enjoy Papers from the Lost Culture of Array Languages.)

read more