SCTP in Go

Rejoice! There seem to be some interest in my SCTP in Go coding project. After reading some positive comments on Reddit I decided to pick up the work where I left it.

The current state of the project is a bare minimum implementation of the protocol in Go 1.6 in my Github account together with some examples. My first task will be to get it working in Go 1.7.

Here a small preview of a SCTP client written in Go:

package main
import (
	"log"
	"os"
	"net"
)

func main() {

	saddr := "127.0.0.1:4242"
	addr, err := net.ResolveSCTPAddr("sctp", saddr)
	if err != nil {
		println(err)
		os.Exit(-1)
	}

	conn, err := net.DialSCTP("sctp", nil, addr)
	if err != nil {
		println("Error listening " + err.Error())
		os.Exit(-1)
	}

	defer conn.Close()

	var message = "hello"
	bmessage := []byte(message)

	_, err = conn.WriteTo(bmessage, addr)
	if err != nil {
		log.Printf("WriteTo error: %v", err)
	}

}

 

More info about SCTP in Go can be found in my paper I published for my talk at the 2013 AsiaBSD conference or a video recording of the talk I gave at the 2012 EuroBSD conference.

How to test a JPA repository

Here is a typical (Hibernate) JPA repository for saving an entity, a Cookie in this case:

@Entity
@Table(name = "cookie")
public class Cookie {

@Id
 @GeneratedValue(strategy = GenerationType.AUTO)
 private Integer id;

@Temporal(TemporalType.TIMESTAMP)
 @Column(name = "last_modified")
 private Date lastModified;

}

@Repository
public class CookieRepository {

@PersistenceContext
 EntityManager em;

@Autowired
 TimeService timeService;

@Transactional
 public Cookie createEntity(Cookie cookie) {
 cookie.setLastModified(timeService.getTime().toDate());
 em.persist(cookie);
 return cookie;
 }
}

Writing a unit test for this is not as straightforward as you might think. When a new cookie is created the id is null and only after it gets persisted in the database it gets assigned a value generated by the database.

The EntityManager persist method does not return a value, so how can we mock the entity manager _and_ set the id of the cookie?

Here is the answer:


@RunWith(MockitoJUnitRunner.class)
public class CookieRepositoryTest {

@Mock
 EntityManager em;

@Mock
 TimeService timeService;

@InjectMocks
 CookieRepository underTest = new CookieRepository();

@Test
 public void testCreateEntity() throws Exception {
 Cookie newCookie = new Cookie();

when(timeService.getTime()).thenReturn(new DateTime(DateTimeZone.UTC));

doAnswer(new Answer<Cookie>() {
 @Override
 public Cookie answer(InvocationOnMock invocationOnMock) throws Throwable {
 Object[] args = invocationOnMock.getArguments();
 Cookie cookie = (Cookie) args[0];
 cookie.setId(1);
 return null;
 }

}).when(em).persist(any(Cookie.class));

Cookie persistedCookie = underTest.createEntity(newCookie);
 assertNotNull(persistedCookie.getId());
 }

}

Mockito comes with the doAnswer method, specifically designed for dealing with void methods. With this method you can catch the input of the void method, persist() in this case, manipulate the object which than can be asserted for correctness later in the test.

A Scala Fractal

As a follow up from my previous Fern Fractal written in Go, here the same written in Scala;
The fractal is a Barnsley Fern and is an example of an iterated function system.

import java.awt.{Color, Graphics2D, Dimension}
import swing.{SimpleSwingApplication, Panel, Frame}
import util.Random
class FernFractalFrame(transformFunction: (Double, Double) => (Double, Double), val width: Int, val height: Int, val max: Int) extends Frame {

contents = new Panel {
 preferredSize = new Dimension(width, height)
 opaque = true

override def paint(g: Graphics2D) {
 g.setBackground(new Color(0, 0, 0))
 g.setColor(Color.GREEN)
 g.drawLine(height / 2, width / 2, height / 2, width / 2)
 drawFern((0, 1), max, g)
 }

def paintPoint(p: (Double, Double), g: Graphics2D) = {
 val scale = height / 11
 val y = (height - 25) - (scale * p._2)
 val x = (width / 2) + (scale * p._1)
 g.drawLine(x.toInt, y.toInt, x.toInt, y.toInt)
 }

def drawFern(p: (Double, Double), max: Int, g: Graphics2D) {
 paintPoint(p, g)
 repaint()
 if (max != 0)
 drawFern(transformFunction(p._1, p._2), max - 1, g)
 }
 }
}

object FernFractal extends SimpleSwingApplication {

object TransformFunction extends CFernTransformFunction

class CFernTransformFunction extends ((Double, Double) => (Double, Double)) {

def rnd = Random.nextInt(100)

def transformPoint(p: (Double, Double), a: Double, b: Double, c: Double, d: Double, s: Double): (Double, Double) =
 ((a * p._1) + (b * p._2), ((c * p._1) + (d * p._2) + s))

def apply(x: Double, y: Double) = {
 rnd match {
 case n if n <= 1 => transformPoint((x, y), 0.0, 0.0, 0.0, 0.16, 0.0)
 case n if n <= 7 => transformPoint((x, y), 0.2, -0.26, 0.23, 0.22, 1.6)
 case n if n <= 14 => transformPoint((x, y), -0.15, 0.28, 0.26, 0.24, 0.44)
 case n if n <= 100 => transformPoint((x, y), 0.85, 0.04, -0.04, 0.85, 1.6)
 }
 }
 }

def top = new FernFractalFrame(TransformFunction, 400, 400, 10000)
}

To run this example:

$> scalac FernFractal.scala
$> scala -cp . FernFractal

Varnish reordering query string

(Update: now listed as a module on the official Varnish site)
(Update: this code is being used in production without any problems in several companies I worked for)

In Varnish the URL is the key to the caching. If it recognises a previously requested URL it will look if it’s available in its cache and deliver this back.  There is a small problem with URLs which have parameters. Take a look at the following queries:

http://localhost/test?ddd=444&bbb=222&ccc=333&aaa=111
http://localhost/test?ccc=333&aaa=111&ddd=444&bbb=222
http://localhost/test?bbb=222&ccc=333&ddd=444&aaa=111

Each of them will return the same result, the parameters are the same, only the order is different.  Varnish treats each of them as a different query and will, in this case, do three separate requests to the backend and cache all of them.

To deal with this I’ve written a small bit of C code that can be embedded in the varnish configuration file which will order the parameters so URLs with unordered parameters will become the ordered and therefor have an equal cache key.

To follow our example the three URLs all get ordered like this:

http://localhost/test?aaa=111&bbb=222&ccc=333&ddd=444

How it works:
I tokenise the url, put the parameters in a binary tree and do an in order traversal to get them out again. Performance of this method is on average O(n log(n)).

Code can be found here: https://github.com/cyberroadie/varnish-urlsort
It has a FreeBSD license so please feel free to use it. One warning though: I haven’t used it in a live environment, so do take care!!

Olivier

Update: As suggested on the Varnish mailing list it now also compiles as a Varnish module and can be used like this (once installed):

import urlsort;
sub vcl_recv {
  set req.url = urlsort.sortquery(req.url);
}

Update:

Checked memory usage with valgrind, no leaks 🙂

valgrind -v --dsymutil=yes ./urlsort "http://localhost/test?ddd=444&bbb=222&ccc=333&aaa=111"
==66758== 
==66758== HEAP SUMMARY:
==66758== in use at exit: 6,241 bytes in 33 blocks
==66758== total heap usage: 39 allocs, 6 frees, 6,445 bytes allocated
==66758== 
==66758== Searching for pointers to 33 not-freed blocks
==66758== Checked 488,280 bytes
==66758== 
==66758== LEAK SUMMARY:
==66758== definitely lost: 0 bytes in 0 blocks
==66758== indirectly lost: 0 bytes in 0 blocks
==66758== possibly lost: 0 bytes in 0 blocks
==66758== still reachable: 6,241 bytes in 33 blocks
==66758== suppressed: 0 bytes in 0 blocks
==66758== Rerun with --leak-check=full to see details of leaked memory
==66758== 
==66758== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 1 from 1)
--66758-- 
--66758-- used_suppression: 1 OSX107:blah
==66758== 
==66758== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 1 from 1)

Why I’m looking at Go!

I watched a Stanford University lecture with as guest speaker Rob Pike, who is a Principle Engineer at Google. He’s part of the team who developed the computer language Go!.

He made some interesting comments on why specific languages exists. First have a look at these statistics (source TIOBE index November 2011):
Types checked compile or runtime:

Category Ratings Nov 2011 Delta Nov 2010
Statically Typed Languages 63.4% +0.5%
Dynamically Typed Languages 36.6% -0.5%

Statically type languages on top. Clear winners. There is some discussion going round which suggest the TIOBE index is biased towards languages which are hard to program in since people ask more questions about it (number of results returned by a search engine is one of the criteria of the index). This of course assumes a statically type languages is de facto difficult.

An interesting point Rob Pike makes is that most languages build as a reaction to these hard to program statically typed languages happen to be dynamically typed. He says ‘Perhaps as a result, non-expert programmers have confused “ease of use” with interpretation and dynamic typing.’

‘Statically typed’ and ‘hard to program’ are not necessarily linked. They are two separate issues.

If you start using Go! the first thing you notice is the super fast compile time. The type checking is clearly not an issue. Even big programs compile almost instantly.

The implicit variables declaration system is easy too:
var x, y, z int
var c, python, java = true, false, “no!”
or within functions
lala := “i am a string”

There are lots of other reasons why Go! is an interesting language, but not having to deal with the tiresome static vs dynamic type discussion is (for me) a very good reason to have a look at this new language.

The Iron-y

Was it only me who noticed it? Visiting the Moctezuma Aztec special exhibition in the British Museum I noticed it was sponsored by ArcelorMittal. What becomes clear half way through the exhibition, which is a well known fact, is the superiority of the Spanish weapons made of steel. One of the vitrines shows the steel harness and sword next to a wooden Aztec club, I guess they omitted the feather warrior outfits so not to make it even more embarrassing. Obviously the Spanish easily conquered the Aztec territories.

But did you know that ArcelorMittal is the biggest steel manufacturer in the world? And not only that, ArcelorMittal is a merger of a couple of companies one of which was Aceralia, the biggest Spanish steel manufacturer.