So imagine you are writing a svelte component, and adding some styling to it.You might create a class named “panel” and use it in your markup. panel is a pretty generic class name, but svelte will create a hash for your component and make sure that your new panel class only applies to elements that also have the svelte hash as a class. I think this is a really smart way of solving css isolation. But the issue is that it only goes in one direction. If we assume that we are in a complex systems with multiple frameworks fighting for attention, it seems not unlikely that someone else has defined a class named panel. Svelte will of course not modify that component, but since svelte doesn’t change the name of your classes we will inherit the styling of panel defined outside of our component.

The issue is in sveltes defense, not a common one and not one I would like the maintainers to fix. And actually it is not even a bad thing that it works this way because it allows us to import an outdated css framework such as bootstrap and use their classes.

But ok, let us say we want to fix it!We can write a plugin that prefixes all of our classes, so that panel becomes prefix-panel and in the case of global styles, we can “import” them by writing @panel and then in the plugin we strip the @ sign, thus allowing global css styles to affect it

ts
import { vitePreprocess } from '@sveltejs/vite-plugin-svelte'
import translations from "./Scripts/Svelte/translationExtractor.js"
import { compile, parse, walk } from 'svelte/compiler';
import MagicString from 'magic-string';
export default {
  preprocess: [vitePreprocess(),
  {
    name: "class prefixer",
    markup: ({ content, filename }) => {
      let result = new MagicString(content, { filename });
      const ast = parse(content, { filename })
      walk(ast, {
        enter(node, parent, prop, index) {
          const data = {
            ...node,
            text: content.substring(node.start, node.end)
          }
          if (node.type === "Attribute" && node.name === "class") {
            for (let c of node.value) {
              if (c.data) {
                result.overwrite(c.start, c.end, c.data.split(" ").map(c => c.startsWith("@") ? c.substring(1) : ("prefix-" + c)).join(" "))
              }
            }
          }
          if (node.type === "ClassSelector" && !node.name.startsWith("@")) {
            if (node.name.startsWith("@")) {
              result.overwrite(node.start, node.end, "." + node.name.substring(1))
            }
            else {
              result.overwrite(node.start, node.end, ".prefix-" + node.name)
            }
          }
          if (node.type === "Class") {
            const start = node.start + "class:".length
            const end = node.start + "class:".length + node.name.length
            if (node.expression.type === "Identifier" && node.end == end) {
              result.overwrite(start, end, `prefix-${node.name}={${node.expression.name}}`)
            }
            else {
              result.overwrite(start, end, "prefix-" + node.name)
            }
          }
        }
      })
      return {
        code: result.toString(),
        map: result.generateMap()
      };
    },
  }
  ]
}

The heavy lifting of course comes from the great plugin support from svelte and the access to its’ compile/parse methods, and MagicString which is a great package for updating multiple places in a string (a package that I will likely put to use in the future)

This bug relates to ThreeJS and as a 3D framework it of course handles matrixes, and float values.The fact that I explicitly say float values, might tick you of that this bug relates to a rounding error, and you would be correct in thinking so.

Imagine setting the position and rotation of a mesh.

const test = new THREE.Mesh(new THREE.PlaneGeometry(), new THREE.MeshBasicMaterial());
test.position.copy(new THREE.Vector3(-0.0800582263098657,0,-0.003907569688712309))
test.rotation.copy(new THREE.Euler(-1.5707963267948966,0,0, "XYZ"))

Nothing out of the ordinary, and since we are making it easy for us we aren’t manually doing anything with the meshes matrix, and let threeJS do that for us on the next render cycle.And this will likely work! at least it did for me. But it turns out that it only worked for me because I am running a chromium based browser, which runs the V8 js engine.When I tried my code in firefox, running SpiderMonkey, everything was renderer upside down because of a rounding error. Or rather, everything that I projected using a raycast was renderer upside down.

What is more interesting (in my opinion) is that the rounding error is really small, and the diff between V8 and SpiderMonkey came down to about 4e-16 in a cell in the meshes matrix.But 4e-16 can be enough to flip a sign from plus to minus, which it did in my case, resulting in the raycast getting flipped.

So can we fix it?

I made my own solution to this, and it is not pretty, but it works.

ts
private setMatrixPrecisionOnMesh(mesh: THREE.Object3D) {
    const update = mesh.updateMatrix;
    mesh.updateMatrix = function updateMatrix() {
        const precision = 10000000
        const round = (num) => Math.round(num * precision) / precision;
        this.position.setX(round(this.position.x));
        this.position.setY(round(this.position.y));
        this.position.setZ(round(this.position.z));
        this.rotation.set(
            round(this.rotation.x),
            round(this.rotation.y),
            round(this.rotation.z),
            this.rotation.order
        )
        update.call(this, {});
    }
}

Using that method I get numbers that aren’t as precise as before, but I get rid of all rounding differences between firefox and chromium.

It is not unlikely that you have fallen into this trap and wondered why docker is being sold that way, and I am hoping to shed some light on the issue with this post.

Production

OK, lets begin with the place that is the most important, Prod!

In the times before Docker, we threw a bundled package over the wall to the ops team and they moved the package over to a prod server and hit the restart button.For the majority of deployments this worked without issues. But it wasn’t perfect. What happened when for instance you upgraded java version?

Let us assume that you had a legacy system that you upgraded from java 8 to java 11. On your local development environment you installed java 11, and made sure everything was working, then you once again threw the bundle to the ops team, and then it got tricky for them. Of course they too needed to install java 11, and they probably needed to change some service definitions to use java 11 instead of 8. So far so good, it should work, even though it was a bit of a hassle. But then it breaks! And you start questioning why it breaks. The same code works on your machine, how come it doesn’t in prod? Turns out you installed oracle JDK11 on your machine, and you only specified java 11 to the ops team, who went ahead and installed GraalVM, and now production is broken.

This hypothetical might not be the biggest of issues, and it can be mitigated in more than one way. But the fact that the underlying systems aren’t working the same way is a massive issue. Another pretty clear example would be if the prod servers are running linux and you are running mac. They are both unix systems so you have access to a lot of similar tools. And you might think that both have access to the sed command since it is in the path on both machines… but of course they don’t work the same, unless you had installed gsed on mac. Small things like that are why we need docker.

Local dev

Ok, so you have dockerized your production environment, great!And you made a deployment that for whatever reason is not behaving like you expected. So you start debugging it, by checking out the same git commit as prod is running and you run something like docker built -t app . && docker run -it app (build and start the docker image).Some progress bars start flying across the screen and then you get an error message and the build halts.Weird, right? it runs in production so this should work. You ask your desk neighbor to do the same and he says “works on my machine”.At this point you start blaming docker, and question what the point is, since it doesn’t solve the core issue!

But here is the thing: The works on my machine issue can show itself in two places. When running and when building.If you would have skipped the build step and instead just ran docker run -it app:prod it is likely to have worked.Or at the very least it would behave the same as the prod version, but it could of course still crash because of networking issues, access problems, or just plainly wrong environment variables.

So why was it working for your desk neighbor and not for you?There could be tons of reasons for this, but for instance you can have different docker layers in your caches. Or, you could have different files on disk.For example your project could have important but (to git) ignored files, and when you are adding files to your docker layer you can get a diff.

Conclusion

So does docker solve the “works on my machine” problem?

Without a question it does solve that problem in some aspects. It just doesn’t solve it in all places you might wish it does.Docker is in no way a magic bullet, but you can be sure that if you test an image in a test environment and send the same image to production you can be sure that the code will behave the same way and any issues are likely to be related to configuration or other supporting systems.Key point here is to use the same image. If you are rebuilding a new image for production it will in essence run untested code.

You can of course do this with some pretty minor Photoshop skills, but let us dive into three js and do it the hard way instead!

Intro

I will be using svelte and the three js wrapper threlte as that is the most convenient for me. But the concepts should translate pretty well to vanilla three js.

To start of, make sure you have your project setup and installed according to respective instructions.

App

Best practice according to threlte is to separate your canvas and your scene, so let’s do that.

App.svelte
svelte
<script>
  import { Canvas } from '@threlte/core'
  import Scene from '$lib/Scene.svelte'
</script>
<div class="app">
  <Canvas>
    <Scene />
  </Canvas>
</div>

And for the fun of it, we might as well slap on some styling on it

App.svelte
svelte
<style>
  .app {
    width: 100%;
    height: 100%;
    position: absolute;
    left: 0px;
    top: 0px;
    background: radial-gradient(circle, rgb(149, 154, 160) 0%, rgb(94, 100, 105) 100%);
    background-position: 0, 0;
  }
</style>

The Scene

We will do most of the heavy lifting here, but to keep the file succinct let us break it into parts.

Scene.svelte
svelte
<script>
  import Target from './Target.svelte'
  import Camera from './Camera.svelte'
  import Lights from './Lights.svelte'
  let camera
  let mesh
</script>
<Target bind:mesh />
<Camera bind:camera />
<Lights />

I think this is a reasonable split. So we have a component Target to hold the model that we are to paste decals on. We have a Camera component that wraps the three JS camera. And lastly we have all our lighting in a Lights component.

We will add the decals and the controller for that in a bit.

Target

For the sake of it I have modelled I water bottle in blender and exported it to GLTF which works quite well in my experience.

Target.svelte
svelte
<script>
  import { GLTF } from '@threlte/extras'
  export let mesh
</script>
<GLTF
  useDraco="https://www.gstatic.com/draco/v1/decoders/"
  url="/bottle.glb"
  on:create={({ ref }) => {
    mesh = ref.children[1]
  }} />

noteworthy here is that the GLTF loader results in a group, and my code only handles a single mesh. So in a hacky proof of concept I am just picking the second child which is the canister of the bottle. In an ideal world you would either have the code handle group, or a list of meshes and just flatmap everything.

Camera

We want a camera, and we want to be able to rotate it around the target.We could use an OrbitControls but I find it a bit limited, and couldn’t find a way to bind it to middle mouse instead of left click, that we will use for pasting decals.

so instead let us add this

Camera.svelte
svelte
<script>
  import { T } from '@threlte/core'
  export let camera
  let rotation = 75 //arbitrary number
  export function drag(e) {
    if (isMiddleMouseDown) {
      rotation += e.movementX
      rotateCamera()
    }
  }
  function rotateCamera() {
    const sensitivity = 100
    const distance = 10
    const pos = [Math.sin(rotation / sensitivity), 1, Math.cos(rotation / sensitivity)].map(comp => comp * distance)
    camera.position.set(...pos)
    camera.lookAt(0, 1, 0)
    camera = camera //force svelte re-render
  }
  let isMiddleMouseDown = false
  function mouseDown(e) {
    isMiddleMouseDown = e.button === 1 //1 is middle mouse
  }
  function mouseUp(e) {
    isMiddleMouseDown = false
  }
</script>
<svelte:window on:mousedown={mouseDown} on:mouseup={mouseUp} on:mousemove={drag} />
<T.PerspectiveCamera
  makeDefault
  bind:ref={camera}
  on:create={() => {
    rotateCamera()
  }} />

Lights

Ok last thing before we start working on the decals.We of course want to add lights to our target.

Lights.svelte
svelte
<script>
  import { T } from '@threlte/core'
  let l1Rot = 1
  let l2Rot = -1
</script>
<T.AmbientLight />
<T.DirectionalLight position={[Math.sin(l1Rot) * 10, 10, Math.cos(l1Rot) * 10]} castShadow intensity={0.8} />
<T.DirectionalLight position={[Math.sin(l2Rot) * 10, 10, Math.cos(l2Rot) * 10]} intensity={0.2} />

Note that this has similar math as the camera to rotate the position of lights around the world origin.

Partial result

We should have something looking like this now

Raycast

Now the fun begins!We want to figure out if the mouse is hovering on our mesh and where.So with some code that is shamelessly stolen and modified from here

Scene.svelte
js
let lineGeometry = new THREE.BufferGeometry()
lineGeometry.setFromPoints([new THREE.Vector3(), new THREE.Vector3()])
let line
const raycaster = new THREE.Raycaster()
const mouseHelper = new THREE.Mesh(new THREE.BoxGeometry(1, 1, 10))
function checkIntersection(x, y) {
  if (mesh === undefined) return
  const intersection = {
    intersects: false,
    point: new THREE.Vector3(),
    normal: new THREE.Vector3()
  }
  const mouse = new THREE.Vector2()
  mouse.x = (x / window.innerWidth) * 2 - 1
  mouse.y = -(y / window.innerHeight) * 2 + 1
  raycaster.setFromCamera(mouse, camera)
  const intersects = []
  raycaster.intersectObject(mesh, false, intersects)
  if (intersects.length > 0) {
    const p = intersects[0].point
    mouseHelper.position.copy(p)
    intersection.point.copy(p)
    const n = intersects[0].face.normal.clone()
    n.transformDirection(mesh.matrixWorld)
    n.multiplyScalar(10)
    n.add(intersects[0].point)
    intersection.normal.copy(intersects[0].face.normal)
    mouseHelper.lookAt(n)
    const pos = line.geometry.attributes.position
    pos.setXYZ(0, p.x, p.y, p.z)
    pos.setXYZ(1, n.x, n.y, n.z)
    pos.needsUpdate = true
    line = line
    intersection.intersects = true
    intersects.length = 0
  } else {
    intersection.intersects = false
  }
  return intersection
}
</script>

this will send out a imaginary line from the center of the camera and forwards into the world, targeting the x,y position of the mouse. If that line intersects our mesh we will get a list with places where it intersects. From that list we can fetch the coordinate in 3d space as well as the normal of the face. The normal of a polygon or a face is a normalized vector that indicates where the it is facing.

This code introduces a few more variables, so let us put them to use, and add a listener for mouse movement

Scene.svelte
svelte
  function mouseMove(e) {
    const intersection = checkIntersection(e.clientX, e.clientY);
  }
</script>
<svelte:window on:click={click} on:mousemove={mouseMove} />
<T.Line bind:ref={line} geometry={lineGeometry} />

This should give us a line pointing to the target, so that can verify that everything is working.

Note that the mouseHelper is not added to the scene. We only have it to make it easier to get the rotation of the decal.

Decals

Now we know how to point to an object, we want to add the decals objects to it.

so let us add some code for that in the scene.

Scene.svelte
svelte
<script>
  import Decals from './Decals.svelte'
  let decalRef
  function click(e) {
    const intersection = checkIntersection(e.clientX, e.clientY)
    if (intersection.intersects) {
      const rotation = new THREE.Euler().copy(mouseHelper.rotation)
      decalRef.addDecal(new THREE.Vector3().copy(intersection.point), rotation)
    } else console.log('no intersect')
  }
</script>
<svelte:window on:click={click} on:mousemove={mouseMove} />
<Decals bind:this={decalRef} {mesh} />

With this we add a new component and tell it to add a decal whenever we click on the target.

Decals.svelte
svelte
<script>
  import { DecalGeometry } from 'three/addons/geometries/DecalGeometry.js'
  import { T } from '@threlte/core'
  import * as THREE from 'three'
  export let mesh
  const textureLoader = new THREE.TextureLoader()
  const decal = textureLoader.load('/favicon.png')
  const decalMat = new THREE.MeshPhongMaterial({
    depthWrite: false,
    polygonOffset: true, //Without this the polygons are overlapping the target and causing artifacts
    polygonOffsetFactor: -4,
    map: decal,
    transparent: true //if the decal is a png with transparency this is needed
  })
  export function addDecal(position, rotation) {
    const material = decalMat.clone()
    const size = new THREE.Vector3(1, 1, 1)
    const m = new THREE.Mesh(new DecalGeometry(mesh, position, rotation, size), material)
    m.renderOrder = decals.length // give decals a fixed render order
    decals = decals.concat([m])
  }
  let decals = []
</script>
{#each decals as decal}
  <T.Mesh is={decal} />
{/each}

This code is pretty short, and to the point. Most of the heavy lifting is done by the DecalGeometry object.

Result

Assuming that everything has been copy pasted correctly it should look something like this.

If not, I suggest having a look at my source code on github.

Lastly if you want to see it in action you can try it out here

Besides the topic of this blog post, the last time for me was svelte. It made me think “why are we shipping massive frameworks to browser”. For those who don’t know svelte is a compiler that takes your frontend code and bundles it into a page. And the code you write feels like it is written in a normal framework, but instead of shipping hundreds of kB or mB of just framework code, Svelte bundles into a small package with only the code needed to run your website.

This post isn’t about svelte though, it is about the latest piece of tech that blew my mind away.Namely wing.

Wing

So what is it about wing that made me go “Wow!“?

Wing is the first language I’ve seen that covers the full stack and is truly cloud native.What I mean by that is that the code you write can be compiled into a terraform script that deploys your app to AWS (or other clouds). This sounds like such a minor thing, because writing terraform scripts isn’t that painful and separating infrastructure as code from normal code doesn’t sound that bad. But what wing does, which I find amazing is that it removes the concept of IaC being different from just code.

Hello World

It might be easiest if I show a basic example of how it works.

bring cloud;
let counter = new cloud.Counter();
let api = new cloud.Api();
api.get("/hello", inflight (): cloud.ApiResponse => {
    counter.inc();
    return cloud.ApiResponse {
        status: 200,
        body: "world ${counter.peek()}"
    };
});

This hello world example will create an api gateway with a lambda that responds to /hello.For each request it will update a counter stored in dynamoDB and respond with world x, where x is the current value of the counter.

This example in itself should be enough to convince you that wing is a pretty cool piece of tech to consider.But lets take it a few steps further!

By running this command

bash
wing it main.w

you start a local simulation of your code (think localstack, but slimmer).And you get a pretty nice UI that shows you how everything is connected, and you can click around to see your state, and execute parts of your code.

Running things locally is in my opinion one of the biggest downsides of going cloud… or rather, the fact that it is difficult to do so.At my previous employments I’ve used localstack quite a lot, because without it I have no idea how my code will react before I release it. And hopefully I haven’t released it to prod yet.

So being able to click around like this is a huge win in my books!

But why stop with being able to simulate your local cloud?Let us add a test and you will see how powerful this simulation is

bring cloud;
bring http;
let counter = new cloud.Counter();
let api = new cloud.Api();
api.get("/hello", inflight (): cloud.ApiResponse => {
    counter.inc();
    return cloud.ApiResponse {
        status: 200,
        body: "world ${counter.peek()}"
    };
});
test "counter increments per request" {
    assert(http.get(api.url + "/hello").body == "world 1");
    assert(http.get(api.url + "/hello").body == "world 2");
    assert(http.get(api.url + "/hello").body == "world 3");
}

Pretty simple, right?And this tests that if you send an HTTP request to /hello a lambda function will run and update dynamodb with each request.That is not something I would consider easy to test normally, unless you first deploy it to aws or localstack and then run an integration test against that.

I am also a massive fan of having the test code in the same file as the real code. It elevates them to first class citizens, which they rightfully are.

Adding complexity

Let us assume that doing updates to the counter is pretty slow, and we want to do that asynchronously.

bring cloud;
let counter = new cloud.Counter();
let queue = new cloud.Queue();
let api = new cloud.Api();
queue.setConsumer(inflight () => {
    counter.inc();
});
api.get("/hello", inflight (): cloud.ApiResponse => {
    queue.push();
    return cloud.ApiResponse {
        status: 200,
        body: "world ${counter.peek()}"
    };
});

Can you imagine doing this with javascript and terraform?It is pretty insane how much power you are getting out of so little code.

Inflight vs preflight

Wing manages to be this cool by utilizing what they call inflight and preflight code.You might have notices the inflight tag in the examples above. It indicates that the code should be deployed and run in the cloud. Everything else is only run during setup.But the fact that you can access preflight resources from an inflight context makes the code feel pretty seamless and you don’t realize that you are writing IaC.

Downsides

No language is perfect.Winglang is no exception to that rule. It is very much still in beta, and it shows. It feels like it has a lot and what it has shows a lot of potential, but it does not feel like a language that is mature enough to be in production yet.

One example for that is this:at time of writing they support noSQL like dynamoDB via the “external libraries”. Great! I didn’t expect seeing a relational database this early in development. But this library is really limited. You can do insert, delete, and get just like expected, but if you don’t know the key, you are stuck with a full scan as the .list() method takes no argument you are getting the entire table dumped.

I should be clear and say that this might be something that could be solved at any time as the language is undergoing some pretty heavy development.

Conclusion

I for one am hoping that they continue with the winglang project, as it shows so much potential. And I will be keeping my eyes open in hopes of seeing the first 1.0 version.

Kanlang

So for those that haven’t followed this blog, I am talking about my programming language Kanlang.It is a language where function names and direct function invocations plain and simple aren’t part of the language.I don’t what to repeat myself to much so either check out the language here or read more in my previous posts

Takeaways

OK, so what have I learnt from this journey?

I learnt that you should implement your own parser. I started out using nearley and while it is a great tool, it fell a bit short when the complexity grew. After replacing it with my own implementation of earley parser everything started flowing so much smoother. And in the end I got a much better understanding of how a parser works. Worth noting that if you are creating a “real language” unlike the POC that I created, you might want to consider something other than earley as it isn’t the fastest algorithm out there (but it handles grammar ambiguity and both left and right recursion).

Going slow is going fast. More than once did I take what looked like minor shortcuts to solve the immediate problem I was facing. It made the project move forwards and I was pretty happy with that, but more often than not did it explode in my face later on, and it because a mess of tangled code. I think it would have been a lot faster to simply take a breath and take my time with each and every problem, no matter how small.

However my most important takeaway is that I am apparently really into this stuff. Writing this compiler has been a ton of fun and now all of a sudden I’ve added a hammer to my toolbelt and everything is starting to look an awful lot like a nail.

For example in pascal it is pretty succinct with the assignment operator := and the reassignment operator =.

pascal
a :=3
a = 2

In older versions of java you had to write out your type in front of the variable name to create it.

java
int foo = 3

I don’t think this makes much sense, as the right hand side is of course already a type, so the type of foo can be inferred from the right side of the operator. More than that it is of course also enforced, so this is not semantically valid.

java
int foo = "bar"

The modern java developer will be quick to point out that this is a thing of the past as the var keyword was introduced in java 10

java
var foo = "bar"

Which allows the type of foo to resolve to the type of the right hand side of the assignment.I think this is great, but my experience is that it hasn’t been met with the reception it deserves, and instead the adoption has been somewhat hesitant.

On to javascript

JS is a beast when it comes to this.It has no less than 3 assignment keywords, var, let, and const.

let and const are similar to each other. The only difference is that const, like you could guess is a constant and thus can’t be reassigned. Note here that it does not mean it is immutable, i.e. this is valid.

js
const a = {foo: "bar"}
a.foo = "hello"

On a slight side note this can be done in java as well with the keyword final

java
final int foo = 3

And in fact this final is inferred by the compiler if you don’t change the value and don’t add final

var is a bit of a game changer in the fact that it escapes the current scope.For instance this piece of code

js
try {
  var foo = "bar"
} catch (e) {}
console.log(foo)

Is fully valid and prints bar. You can imagine it becomes more interesting if the right hand side could throw errors.If you aren’t using var it would look like this

js
let foo;
try {
foo = "bar"
} catch(e) {}
console.log(foo)

To me I would prefer the var code example.But here is where it becomes an issue.

Standard lint rules for javascript (and typescript) blocks all usage of var, which is crippling the language.

On top of that the standard rules tell you that you have to use const if you variable is not changing, something that is inferred by the typescript/javascript compiler/runtime.

where am I getting with this

OK, so there are different ways of assigning variables, what is my point?The point I am trying to make is that variable assignments in programming should be the same no matter how it is used, or the diff should provide some additional layer of control.

java
final int a = 3

the final part adds a layer of control meaning it shouldn’t be changed. But nothing is forcing it to be there.

The int part however is not providing control and it is just keyboard slavery. I, as a developer have to write int (prior to java 10).

In typescript/javascript the compiler isn’t forcing me to do anything but the lint rules that are everywhere and considered best practice will make me a slave under its’ rules.I have to write either let or const and I have no choice in which one I pick, it is predetermined by the linter.

My own design

In the language i am writing (kanlang) I will use the same type as pascal, i.e. := for the simple reason that it is short and to the point. The type is inferred from the right hand side, and there is no keyboard slavery.For solving the “issue” with having constant variables, I am considering something along the lines of !:= to indicate that we are assigning and we are not allowed to change it

Conclusion

Us developers should be free to make our own decisions of the code, and the compiler should feel free to add optimisations such as implicit final where it doesn’t change any functionality but improves the performance.

Well look no further for I have just the thing for you!

Kanlang

A while back I wrote a post about a conceptual language that I wanted to create, and today I have reached a milestone!The language is barely working, but it is almost a proof of concept.

So what is it all about?

Kanlang is/will be a Object oriented language with a rigid typesystem, where no two functions can have the same return type.

That is the core point of Kanlang, and from that one can create some pretty weird systems. For example, if I create a variable of type A any transformation of that variable to, for instance type B can be done implicity without calling any functions.

Let me give you an example:

js
fn celsiusToFahrenheit(celsius Celsius alias num) Fahrenheit alias num {
    num fahrenheit = celsius * 9 / 5 + 32
    return fahrenheit
}
Celsius c = 100
Fahrenheit f

We have a Celsius type in our scope and we have a function that converts from Celsius to Fahrenheit, so we don’t need to explicitly call said function. It is enough for us to declare that we want Fahrenheit and the language will use whatever we have in our scope to make that happen.

Note that there shouldn’t be anything stopping you from directly calling the celsiusToFahrenheit function directly if you want… except that it is not yet implemented

Proof of concept

So Kanlang is not really done.I think calling it a proof of concept is overselling it a bit as well.Maybe we can go for a pre-proof of concept?

I think that the idea of this language is kind of fun and interesting.And there is a saying that constraints are good for innovation and creativity, so I have a hope that the constraints of Kanlang will lead to something.Of course that something will not be production, but maybe another language somewhere down the lines.

Current and future

Currently there are tons of things not implemented.To name a few:

• Its’ typesystem is not that rigid yet
• It only really supports numbers
• It doesn’t support direct function calls

But it is available in an online editor.It doesn’t run anything but it shows the javascript that it is compiled down to.

Advent of code

I have so far had a lot of fun creating this language, so I will not stop here.My plan is to continue to develop it into something that actually works.And to make things a bit more fun for me I will start doing advent of code challenges with the language, and fix whatever feature I need in order to pass each challenge.

I am guessing it is going to take me more than a month to do that though, so wish me luck!

Oh, and if you want to have a look at the internals of it, then check out the github repo

Code assignment

This type involves giving the applicant an assignment to develop and submit before the interview to ensure they know how to write code. However, it doesn’t give the interviewer much input. Without knowing the conditions the code was written under, how long it took, and how much help the applicant received, it becomes challenging to gauge their skills.

It should be a starting point for discussions, but it could also be overwhelming depending on the task.

Some people may even turn down offers that require homework, not because they don’t know how to code, but because of the principle of it.

Let’s just talk about it

This approach is way to common. You sit down for an hour with a manager, a developer, and the applicant and talk loosely about what they have worked with.And after the hour is up you go with gut and approve or reject them soley because of their confidence, which it usually boils down to.

The issue with this approach is that it is really hard to gauge what the applicant knows. And I think it is really easy to pass these interviews if you just “sound confident”. On the other side of it, it is really hard as an interviewer to get a feel for how much they actually know.

Create a quicksort algorithm

I’ve never been on either side of this type of interview, where you get a code problem to solve live in front of the interviewers.Which is unfortunate, because I would like to try it. That said I can kind of see a flaw in this approach which is that day-to-day work is just so disconnected for these types of tasks.

My job is basically just pushing data around… with some bells and whistles.I am not doing anything remotely complex enough for me to have to be that strong in algorithmic design. I hope that I am anyways, but as said, I’ve never gotten this type of interview.

How to interview the right way

In my opinion, the most valuable assignment to evaluate someone’s skills is to have them review a piece of code. It doesn’t take more than 30 minutes, and it gives you a pretty good insight into whether they know what they are talking about or not.

So here is how you do it. You create a piece of code in the “target language” that is roughly 50 lines of code. This could be a class, a UI component, a PHP file, or whatever seems reasonable.Then, start adding errors. Add easy syntax errors, such as not referring to this where needed, or null checking before assignments(when you know it to be null).Add some bad practices, like not doing anything in a catch statement. Add some bigger structural issues, such as “should this UI component really be in charge of HTTP requests, or shouldn’t that go through a service layer?”.

Hand them this piece of code, and tell them that it is full of errors.Tell them to treat it as a pull request, and ask if they would be happy if it got released to production.Ask them if they would do something differently.Give them ten minutes to think, where you leave the room and grab a coffee.

When you come back they can give you their findings, and you can discuss what they found. In my experience it is rare for anyone to find all the errors, so you should guide them a bit. For example “what would happen here on line x if y?”. Either they get it at that point or they are completely lost. It becomes a great indicator of whether they know what they are talking about, as you clearly know that a senior developer should have spotted the error straight away, or as a junior dev they should spot it if they are guided a bit towards it.

I’ve turned down quite a lot of developers that sounded really confident, but could not do an easy code review. There is, of course, the counterargument that they might just be nervous or stressed, but I think that if you can’t handle that, I wouldn’t trust you to help out when there is a fire in production.

Conclusion

In conclusion, conducting a code review as part of your interview process can be a valuable tool for evaluating a candidate’s skills and determining if they are the right fit for the job. It allows you to gauge their ability to read code, attention to detail, and understanding of best practices. By following the steps outlined in this post, you can improve your chances of hiring the right developer for your team.

So, what is this revolution I am talking about?Aspect-oriented programming!… I know it is not new, but if you combine it with other paradigms, and drag it to it’s extreme it can become a odd language that someone might enjoy. (emphasis on the “might”)

Introducing Kanlang: The Aspect-Oriented Programming Language

If you’re a software developer, you’ve probably heard of aspect-oriented programming (AOP). AOP is a programming paradigm that focuses on modularizing cross-cutting concerns, such as logging, caching, and security, into separate units called “aspects.” By separating these concerns from the rest of the code, AOP helps developers write cleaner, more maintainable code.

Here are some of the key features of Kanlang:

Unique Return Types

In Kanlang, no two functions can have the same return type. This might sound like a limitation at first, but it’s hopefully a powerful feature that helps you write more expressive code. With unique return types, you can see at a glance what each function does and what it returns. And if you ever need to change the behavior of a function, you can do so without worrying about breaking other parts of the code.

And I kind of hear you saying that it is going to be an issue, because functions need to be able to return the same object.And you are somewhat right. With Kanlang I am planning on adding type alias, meaning two objects can be virtually the same, but named differently. An authentication function could take in a User object, and return a AuthenticatedUser object.

Dependency Injection at the Core

Dependency injection (DI) is a design pattern that makes code more modular and testable by decoupling the components of a system. In Kanlang, DI is not just a design pattern - it’s a core feature of the language. With Kanlang, you can inject variables wherever you need them, even inside code blocks. And because Kanlang uses static typing, the compiler can check that the injected variables are of the correct type.

It can even go as far as to check the method arguments that the providing function requires.Meaning that inside of a code block you can inject an AuthenticatedUser without explicitly calling the function, assuming that you have a UnathenticatedUser object somewhere in your scope.

Conclusion

This is going to be an extremely odd language, that is likely never to see the light of production, but I see it as a interesting thought experiment.

And to give you a taste of what I am envisioning here is a code snippet.

type User {
    email: string
    password: string
}
fn authUser(User u) AuthenticatedUser alias User {
    if u.email == 'abc@email.com' && u.password == 'abc123' {
        return u
    }
    throw 'User can't authenticate'
}
fn welcomeUser(User u) UserGreeting alias string {
    *AuthenticatedUser authedUser //Let the star indicate that it is supplied through DI, and it should "resolve" the call to authUser using u
    return "hello " + authedUser.email
}
User myUser = {
    email: 'abc@email.com',
    password: 'abc123'
}
print(*UserGreeting)

Lately I’ve been using github pages a lot, and I would recomend anyone who isn’t using it to start using it.It saves you the hassle of having to get hardware to host a server, scaling if you get a lot of users (not that I do), and general avalability which you usually don’t get with your run of the mill at home closet servers.

In fact I host this blog on GH pages and it is dead simple with a github action.Here is the one I use to push this blog to GH pages

.github/workflows/main.yaml
yaml
name: CI
on:
  push:
    branches: [main]
# Sets permissions of the GITHUB_TOKEN to allow deployment to GitHub Pages
permissions:
  contents: read
  pages: write
  id-token: write
# Allow one concurrent deployment
concurrency:
  group: 'pages'
  cancel-in-progress: true
jobs:
  home:
    runs-on: ubuntu-latest
    environment:
      name: github-pages
      url: ${{ steps.deployment.outputs.page_url }}
    steps:
      - uses: actions/checkout@v3
      - name: Use Node.js 18
        uses: actions/setup-node@v2
        with:
          node-version: 18
      - name: npm install and build
        run: |
          npm install
          npm run build
      - name: Setup Pages
        uses: actions/configure-pages@v2
      - name: Upload artifact
        uses: actions/upload-pages-artifact@v1
        with:
          # Upload entire repository
          path: './build'
      - name: Deploy to GitHub Pages
        id: deployment
        uses: actions/deploy-pages@v1

But this post is not really about GH pages, it is rather about what to do when you can’t use it.

Serverless backends

So I was writing a time reporting page, because who doesn’t love to do time reporting in tons of places!The plan was to make it sync with github gists, but the problem I ran into was that my frontend couldn’t read gists without cors issues.The solution to this was to host a server that can proxy the requests to the frontend.

Now the issue with that is that I have a bad habbit of creating hobby projects that plain and simply wastes resources by being up all the time.It might not be a lot, since a node server with no traffic doesn’t eat a lot of ram and it doesn’t take that much CPU, but it still hurts me a bit to have them just lay there doing nothing.

What I could do to mitigate this is just do a “regular” kubernetes deployment and scale it down to zero when it is not needed, but I fear that one day when I do need it, and I have no access to my cluster to scale it up.

The sane approach is to start using AWS lambda, Azure functions or google cloud functions, as they also scale to zero and cost next to nothing, with the addition of being highly available… but that is not the approach I went with. Instead I installed knative in my k8s cluster.

Knative

Knative is a k8s controller which gives you “serverless” properties on your k8s cluster. Of course it is not serverless as you need your own k8s servers, but you get the point.You can map a domain to a docker image and knative will spin that container up for you when needed, and kill it after a set time of inactivity.

The deployments are quite easy to setup as well, assuming you aren’t doing anything fancy

yaml
apiVersion: serving.knative.dev/v1
kind: Service
metadata:
  name: timestat
  namespace: timestat
spec:
  template:
    spec:
      containers:
        - image: ghcr.io/munhunger/timestat/kit:{{ .Values.version }}
          ports:
            - containerPort: 3000

That piece of config is all that is needed to get a node container available at https://timestat.home.wunderdev.com/, which will get get started when needed and killed after in my case 10 minutes of inactivity.

microk8s

I was planning on writing this a while ago, but I got into some issues with upgrading my k8s cluster to support a specific version of knative with auto TLS support.And as it was my third time bricking my k8s cluster during the upgrade I started looking elsewhere as there has to be some k8s installer with sensible defaults.That search led me to my new favorite piece of software, namely microk8s.

It might not be for production, but for my home setup it is amazing.Dead simple to install, and it has a pretty decent plugin support (knative among others), so you can quickly spin up a node with some cool k8s controllers and get going within an hour!That is pretty mind boggling, as it usually takes me hours if not days to setup a working k8s cluster (I still have issues with the networking layer).

Just remember if you are using microk8s to create an ingress towards your knative network gateway. That did cause me some headache.

yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: kourier-ingress
  namespace: knative-serving
  annotations:
    cert-manager.io/cluster-issuer: lets-encrypt
spec:
  tls:
    - hosts:
        - '*.home.wunderdev.com'
      secretName: home-tls
  rules:
    - host: '*.home.wunderdev.com'
      http:
        paths:
          - backend:
              service:
                name: kourier
                port:
                  number: 80
            path: /
            pathType: Prefix

Without it, I had nginx loadbalancer serve all my network request, and it of course had no concept of knative

Well, there is an algorithm that cleverly has the same name as that phenomena. And more likely than not, you have used it before.Imagine solving a completely empty sodoku. When you start out all cells can be all values, but as soon as you enter a value somewhere(at random) the wave collapses and you impose rules on other cells.Following these easy steps you can collapse the entire board into a “valid” state.

So the “only” thing to wave function collapse is to figure out what those rules are. Like in sodoku where each number can only appear once per row, per column, and per 3x3 square.

Getting into it

In this blog post we will use wave function collapse to generate a small factory room.And we will start by looking at these four components that I got from Kenney.

We can setup some easy rules for this, for example:

• a box needs to be next to at least one conveyor
• a conveyor can not be horizontally adjacent to a floor
• the conveyor starting point needs to be adjacent to a conveyor
• the floor can go anywhere

So, lets create a six by six grid to play around with.

Notice the numbers on top of each cube, they represent the amount of possible components at each square.To begin with everything is valid, but try clicking a center cube and selecting a conveyor.

You will notice that the numbers update and to the left and right of the conveyor it says 3, since our rules do not allow us to have a floor next to a conveyor.

If you keep selecting valid components you will sooner or later see that adjacent cubes gets updated since there is only one valid option.

Automation

Clicking around in a janky UI like this might be fun and all, but it does get tiresome, so let me introduce a button that just picks a random square and a random valid component for us.

This is a very simple implementation and you might have noticed that it did not manage to “solve” the scene and ended up with one or more squares where nothing could be placed.Once again it works like sodoku. You do your best and in some scenarios you need to guess and see where that leads you. If it doesn’t work, then you backtrack to the stage where your last guess where… I know that the true sodoku pro solvers don’t do any guessing, so this might just be my way of doing it. In any case this is generally how you do it with wave function collapse as well. If you end up in an unsolvable state you should backtrack a bit and try again.

We will not go into details on backtracking in this post.

But I want to show the code of this basic wave function collapse

entities.ts
ts
function checkHorizontal(ctx: { scene: Array<Array<{ type: string }>>, x: number, y: number }, type: Array<string>, depth = 99, strict: boolean = false) {
    let offset = 1
    let count = 0
    if (!strict)
        type.push("interactive") //since this is a wildcard it should be valid (I call an empty square "interactive" in my code for some reason)
    while (type.includes(((ctx.scene[ctx.x + offset] || [])[ctx.y] || {}).type) && offset <= depth) {
        count++
        offset++
    }
    offset = -1
    while (type.includes(((ctx.scene[ctx.x + offset] || [])[ctx.y] || {}).type) && -offset <= depth) {
        count++
        offset--
    }
    return count
}
export abstract class Entity {
    constructor(public x: number, public y: number) {
    }
    abstract isInvalid(scene: Array<Array<{ type: string }>>): boolean
    abstract name: string
}
export class Box extends Entity {
    name = 'box'
    isInvalid(scene: { type: string; }[][]): boolean {
        return checkHorizontal({ scene, x: this.x, y: this.y }, ['conveyor'], 1) < 1
    }
}
export class Conveyor extends Entity {
    name = 'conveyor'
    isInvalid(scene: { type: string; }[][]): boolean {
        return checkHorizontal({ scene, x: this.x, y: this.y }, ['box', 'conveyor', 'conveyor-start'], 1) !== 2 ||
            (checkHorizontal({ scene, x: this.x, y: this.y }, ['conveyor-start'], 1, true) >= 2)
    }
}
export class ConveyorStart extends Entity {
    name = 'conveyor-start'
    isInvalid(scene: { type: string; }[][]): boolean {
        return checkHorizontal({ scene, x: this.x, y: this.y }, ['conveyor'], 1) < 1
    }
}
export class Floor extends Entity {
    name = 'floor'
    isInvalid(scene: { type: string }[][]): boolean {
        return false
    }
} 

As you can see I framed the rules as “what is not valid”. This way I can try to place for example an Conveyor in a square and check if the entire board is valid. If it is not, then that Conveyor cannot be placed in that square.

Worth noting that nowhere do I state that this is the way to do it, or that this is in any way performant. It is just the first hacky solution that I got working, and we will update these rules further on in this post.

In a real world scenario you can expand a lot on these rules. For example you can generate a room, and that room would be invalid if it did not have a door, which would force the algorithm to place a door.

But in this post we will stick to simple adjacency rules.

wfc.js
js
function step() {
  do {
    updateAllowList()
  } while (solveForced() > 0)
}
function cloneScene() {
  return JSON.parse(JSON.stringify(scene))
}
function updateAllowList() {
  scene.forEach((col, x) =>
    col.forEach((entity, y) => {
      if (entity.type !== 'interactive') return
      const allowList = [new Box(x, y), new Floor(x, y), new Conveyor(x, y), new ConveyorStart(x, y)]
        .filter(type => {
          let sceneClone = cloneScene()
          sceneClone[x][y].type = type.name
          let valid = true
          for (let xX = 0; xX < sceneClone.length; xX++) {
            for (let yY = 0; yY < sceneClone[xX].length; yY++) {
              let active = [new Box(xX, yY), new Floor(xX, yY), new Conveyor(xX, yY), new ConveyorStart(xX, yY)].find(
                v => v.name == sceneClone[xX][yY].type
              )
              if (active && active.isInvalid(sceneClone)) valid = false
            }
          }
          return valid
        })
        .map(v => v.name)
      solver[x][y] = assets.filter(v => allowList.includes(v))
    })
  )
}
function solveForced() {
  let solvedCount = 0
  solver.forEach((col, x) =>
    col.forEach((cell, y) => {
      if (cell.length === 1 && scene[x][y].type == 'interactive') {
        if (rules[cell[0]]) {
          rules[cell[0]].apply({}, [{ scene, x, y, checkHorizontal, updateAllowList }])
        }
        scene[x][y].click({ scene, entity: scene[x][y], x, y, asset: cell[0] })
        solvedCount++
      }
    })
  )
  return solvedCount
}

So in essence what we do is, we go through the entire scene and check which entities are valid for each square.If we find one or more places where only one entity is valid, we place it and repeat.

Automating the rules

Since the rules we specified are so simple, I think we can automate the creation of it.The way to do it is to take an example hand crafted scene and check what objects are allowed next to eachother.

It is a bit tedious to build one of these input scenes, so I’ve spared you from having to do it and we will use this scene that I threw together as a baseline

And for reference this is how I decided to represent it

wfc.js
js
//<entity name>:<rotation of the entity in 90 degree increments>
export let defaultScene = [
  ['wall-corner:0', 'wall:1', 'wall:1', 'door:1', 'wall:1', 'wall:1', 'wall:1'],
  ['wall:0', 'floor:0', 'floor:0', 'floor:0', 'floor:0', 'box:0', 'floor:0'],
  ['wall:0', 'box:0', 'floor:0', 'floor:0', 'floor:0', 'conveyor-open:0', 'floor:0'],
  ['door:0', 'floor:0', 'floor:0', 'floor:0', 'floor:0', 'conveyor:0', 'floor:0'],
  ['wall:0', 'floor:0', 'floor:0', 'floor:0', 'floor:0', 'conveyor-cover:0', 'floor:0'],
  ['wall:0', 'floor:0', 'floor:0', 'floor:0', 'floor:0', 'conveyor-open:0', 'floor:0'],
  ['port:0', 'floor:0', 'conveyor-start:1', 'conveyor:1', 'conveyor-open:1', 'conveyor-open:0', 'floor:0'],
  ['wall:0', 'floor:0', 'floor:0', 'floor:0', 'floor:0', 'floor:0', 'floor:0'],
  ['wall:0', 'conveyor-start:1', 'conveyor:1', 'conveyor:1', 'conveyor-cover:1', 'conveyor-open:1', 'box:0'],
  ['wall:0', 'floor:0', 'floor:0', 'floor:0', 'floor:0', 'floor:0', 'floor:0'],
  ['wall-corner:3', 'wall:3', 'wall:3', 'wall:3', 'wall:3', 'wall:3', 'wall:3']
]

So lets’ see what rules we can manage to extract from this scene. We will begin by simply counting how many times entities occur next to each other. For example how often is there a conveyor to the left and right of a box.

wfc.js
js
  function unique(value, index, self) {
    return self.indexOf(value) === index
  }
  export let scene = defaultScene.map(v => v.map(a => ({ type: a.split(':')[0], rot: a.split(':')[1] })))
  let adjacencyCount = {}
  /**
   * Take the default scene and map it into a list of unique assets
   * Have each asset represent a list of all other unique assets with a horizontal and vertical count
   */
  defaultScene
    .flat()
    .filter(unique)
    .forEach(asset => {
      adjacencyCount[asset] = {}
      defaultScene
        .flat()
        .filter(unique)
        .forEach(other => (adjacencyCount[asset][other] = { horizontal: 0, vertical: 0 }))
    })
  for (let x = 0; x < defaultScene.length; x++) {
    for (let y = 0; y < defaultScene[x].length; y++) {
      // we will only use manhattan adjacency so diagonals are irrelevant
      for (let xX = -1; xX <= 1; xX += 2) {
        if (x + xX >= 0 && x + xX < defaultScene.length)
          adjacencyCount[defaultScene[x][y]][defaultScene[x + xX][y]].horizontal++
      }
      for (let yY = -1; yY <= 1; yY += 2) {
        if (y + yY >= 0 && y + yY < defaultScene[x].length)
          adjacencyCount[defaultScene[x][y]][defaultScene[x][y + yY]].vertical++
      }
    }
  }

with that code that does not follow industry standards of not having side effects, the adjacencyCount should look something like this

json
{
  "wall:1": {
    "wall-corner:0": {
      "horizontal": 0,
      "vertical": 1
    },
    "wall:1": {
      "horizontal": 0,
      "vertical": 6
    },
    "door:1": {
      "horizontal": 0,
      "vertical": 2
    },
    "wall:0": {
      "horizontal": 0,
      "vertical": 0
    },
    "floor:0": {
      "horizontal": 4,
      "vertical": 0
    },
    "box:0": {
      "horizontal": 1,
      "vertical": 0
    },
    "conveyor-open:0": {
      "horizontal": 0,
      "vertical": 0
    }
  }
}

A bit shortened of course to focus on the important aspects.But that object tells us that there are 6 walls with a rotation value of 1 stacked next to eachother vertically.It also tells us that there is one box in front of a wall.Looking at it the other way around we see that it is not allowed to have a conveyor-open object anywhere near a wall.

I want to extend it a little bit so that we know more than just “is it valid”, and adding a probablity of how likely it is that a box is in front of a wall.

wfc.js
js
  let rules = {}
  Object.keys(adjacencyCount).forEach(entityKey => {
    let entity = adjacencyCount[entityKey]
    // figure out the statistical probability of there being an x horizontally adjacent
    let totalHorizontal = Object.values(entity)
      .map(v => v.horizontal)
      .reduce((acc, val) => acc + val, 0)
    let horizontalRules = Object.keys(entity)
      .map(v => ({
        entity: v,
        odds: entity[v].horizontal / totalHorizontal
      }))
      .filter(v => v.odds > 0)
    let totalVertical = Object.values(entity)
      .map(v => v.vertical)
      .reduce((acc, val) => acc + val, 0)
    let verticalRules = Object.keys(entity)
      .map(v => ({
        entity: v,
        odds: entity[v].vertical / totalVertical
      }))
      .filter(v => v.odds > 0)
    rules[entityKey] = { horizontalRules, verticalRules }
  })

So now we should have a normalized list of possible entities

json
{
  "wall:1": {
    "horizontalRules": [
      {
        "entity": "floor:0",
        "odds": 0.8
      },
      {
        "entity": "box:0",
        "odds": 0.2
      }
    ],
    "verticalRules": [
      {
        "entity": "wall-corner:0",
        "odds": 0.1111111111111111
      },
      {
        "entity": "wall:1",
        "odds": 0.6666666666666666
      },
      {
        "entity": "door:1",
        "odds": 0.2222222222222222
      }
    ]
  }
}

With this it is clear that there is a 1 in 5 that there if a box and not a floor in front of a wall.We will not get any rules saying that if there is a box next to a conveyor, there has to be another conveyor next to that conveyor. But I think this should suffice

Here is the solver I made using the rules extracted from the default scene.

Results

The first solution I got looked like this

Which was 100% user error my part, but I found it pretty funny. It also gave me the idea to bootstrap the solver by placing a corner wall at the corner of the scene.This forced the solver to add some walls, and from time to time it added a lot of walls

But in general the result looked acceptable, albeit far from great

In all honesty I did not expect it to be this far from perfect, but from time to time it produces an acceptable result.And to be fair, it makes perfect sense that it looks like crap, only looking at the immediately adjacent entites will result in a pretty limited ruleset.

If I were to expand on this I would force it to encase the scene in walls, by making a rule that checks the perimiter of the scene and making it invalid if it is not encased.Then I would add rules for minimum amount of objects. For example a room would not be complete without a door, or a box, or a conveyor. Given adjacency rules that would force the door to be placed on a wall and not freestanding in the middle of the room.

Turns out it was not as simple as I expected, but then again, I have no experience at all of this type of stuff.

So for anyone else wanting to get into 3d browser stuff, let the blind lead the blind.

Setting up three js

OK, so three js seems to be some form of standard when it comes to browser 3D stuff. I started using it, but ran into some issues when trying to use physics. Stepping up a level in abstraction I found threlte which is a threejs wrapper for svelte. You still have access to everything threejs, but it comes with sensible defaults and it has an easy to use integration with rapier. With rapier and threlte, physics was a lot easier than running with threejs and ammo or cannon-es

Now that we know what to use and what abstraction level to be at, lets get started and create a basic scene. I think it makes sense to steal from the best, so following the getting started guide at threlte, and mofifying it slightly we get something like this

Canvas.svelte
html
<script>
  import { Canvas } from '@threlte/core'
</script>
<div class="h-96">
  <Canvas>
    <slot />
  </Canvas>
</div>

diceRoller.svelte
html
<script>
  import { OrbitControls, T } from '@threlte/core'
  import { degToRad } from 'three/src/math/MathUtils'
</script>
<T.PerspectiveCamera makeDefault position={[0, 10, 10]} fov={24}>
  <OrbitControls maxPolarAngle={degToRad(80)} enableZoom={false} target={{ y: 0.5 }} />
</T.PerspectiveCamera>
<T.DirectionalLight castShadow position={[3, 10, 10]} />
<T.DirectionalLight position={[-3, 10, -10]} intensity={0.2} />
<T.AmbientLight intensity={0.2} />
<T.Group>
  <T.Mesh position.y={0.5} castShadow let:ref>
    <T.BoxGeometry />
    <T.MeshStandardMaterial color="#333333" />
  </T.Mesh>
</T.Group>
<T.Mesh receiveShadow rotation.x={degToRad(-90)}>
  <T.CircleGeometry args={[3, 72]} />
  <T.MeshStandardMaterial color="white" />
</T.Mesh>

Note that we are using a separate component just for the canvas.The reason for this is so that we later on can use the hook useThrelte that gives us access to the threejs scene graph. It will be more obvious later on why this is needed.

Making it look like a dice

I want to focus this blog on only one dice, and I don’t want to make it easy. So lets focus on a twenty sided dice.

There might a “pre-made” geometry for it, but I’ve exported a one segment ico sphere from blender.If you want to follow along you can download it here.

So now lets load the dice.

diceRoller.svelte
html
<script>
  import { useLoader, OrbitControls, T, Mesh } from '@threlte/core'
  import { OBJLoader } from 'three/examples/jsm/loaders/OBJLoader'
  import { degToRad } from 'three/src/math/MathUtils'
  import { onMount } from 'svelte'
  import { CircleGeometry, DoubleSide, MeshStandardMaterial } from 'three'
  /**
   * @type {import("three").BufferGeometry}
   */
  let d20Mesh
  const loader = useLoader(OBJLoader, () => new OBJLoader())
  onMount(() => {
    loader.load('/assets/d20.obj', obj => {
      d20Mesh = obj.children[0].geometry
    })
  })
</script>
<T.PerspectiveCamera makeDefault position={[0, 10, 10]} fov={24}>
  <OrbitControls maxPolarAngle={degToRad(80)} enableZoom={false} target={{ y: 0.5 }} />
</T.PerspectiveCamera>
<T.DirectionalLight castShadow position={[3, 10, 10]} />
<T.DirectionalLight position={[-3, 10, -10]} intensity={0.2} />
<T.AmbientLight intensity={0.2} />
<T.Group>
  <Mesh
    position={{ x: 0, y: 1.5, z: 0 }}
    castShadow
    geometry={d20Mesh}
    material={new MeshStandardMaterial({ color: '#333333', side: DoubleSide })} />
</T.Group>
<Mesh
  receiveShadow
  interactive
  rotation={{ x: degToRad(-90) }}
  material={new MeshStandardMaterial({ color: 'white' })}
  geometry={new CircleGeometry(3, 72)} />

We load the dice by declaring that we are to use a .obj loader, and asynchronously load it when the svelte component is ready.

And we update the <T.Mesh> component to one of the clearer threlte wrapped <Mesh> components. With this component it is pretty easy to just jack in our loaded obj mesh.

Before moving on to physics and dice rolling, lets throw a texture on the dice so that we can see the value of each face. I hastily made one that you can find here. It does not look good, but it gets the job done.

diceRoller.svelte
html
<script>
  let d20Texture
  const loader = useLoader(OBJLoader, () => new OBJLoader())
  const textureLoader = useLoader(TextureLoader, () => new TextureLoader())
  onMount(() => {
    textureLoader.load('/assets/d20.jpg', texture => {
      texture.magFilter = NearestFilter
      texture.minFilter = LinearMipMapNearestFilter
      d20Texture = texture
    })
  })
</script>
  <Mesh
    position={{ x: 0, y: 1.5, z: 0 }}
    castShadow
    geometry={d20Mesh}
    material={new MeshStandardMaterial({ map: d20Texture, side: DoubleSide })} />

Somewhat abreviated, but pretty straightforwards.In the same way as with the dice we declare that we want to use a TextureLoader, and use it to load our texture jpg. The texture can then be added easily to our dice mesh material.Note that the magFilter and minFilter settings aren’t needed, but it indicates to threejs how it should scale/shrink textures.

Throwing dice

Alright, lets get to the interesting stuff.I wish I could give you more of these “live views” but unfortunately rapier does not play nice when there is more that one physics world loaded.

But lets get into it anyways.

So to begin with we need to wrap our scene in a <World> tag.And then we need some static colliders

diceRoller.svelte
html
  <!--floor-->
  <Collider shape="cuboid" args={[100, 0.1, 100]} />
  <!--top and bottom wall-->
  <Collider shape="cuboid" args={[1000, 100, 1]} position={{ x: 0, y: 0, z: 10 }} />
  <Collider shape="cuboid" args={[1000, 100, 1]} position={{ x: 0, y: 0, z: -7 }} />
  <!--left and right wall-->
  <Collider shape="cuboid" args={[1, 100, 1000]} position={{ x: 10, y: 0, z: 0 }} />
  <Collider shape="cuboid" args={[1, 100, 1000]} position={{ x: -10, y: 0, z: 0 }} />

These colliders are there so that our dice don’t just fall into the void, and they are somewhat contained within our viewport.Without knowing I feel pretty confident in saying that this for sure is not the smartest approach, and you would want something that is easier to adapt to whatever the camera i seeing/the canvas ration. But this works, and it is good enough for experimenting with.

Now that we have the static colliders defined, we need to add a rigid body to our dice.And to clarify a collider is a static object that is not moving and that is not rendered. A rigidbody is affected by physics, can be rendered, and collides with colliders and other rigid bodies.

html
      <RigidBody
        bind:rigidBody={d}
        position={{ x: Math.random(), y: 4, z: 8 }}
        linearVelocity={{ x: Math.random() * 8 - 4, y: Math.random(), z: Math.random() * -20 }}
        angularVelocity={{
          x: Math.random() * 3,
          y: Math.random() * 3,
          z: Math.random() * 3
        }}>
        <AutoColliders shape="convexHull" mass={0}>
          <Mesh />
        </AutoColliders>
      </RigidBody>

So by wrapping our dice with <RigidBody> and <AutoColliders> it becomes part of the physics simulation.

Note that we are setting a both a linear and an angular velocity on the rigid body, meaning that we will “throw” it with some spin applied.

Additionally and this is very important, you need to remove the position from the mesh and add it soley to the rigid body. We will go into more details about this later.

But with all of this we should have something that looks like this

It is not perfect, but it works and I am pretty happy with it, even though I wouldn’t push it to anything near production ready code.

Reading the dice

This part took me a bit over a day to figure out, but we need to read the face of the dice we just threw.In theory there is a pretty simple way of doing it. Just do a raycast from straight above each dice down into the ground.The raycast should give two faces, the top and bottom one. The bottom one can easily be discarded because it is further from the raycast origin. With the top face you can then cross reference the mesh face index with the face value… sounds simple, so why did it take me so long to do, and why did I say that it is simple in theory?

Well, as I mentioned when adding the <RigidBody> tag it is important to move the position away from the <Mesh> tag. I had overlooked that part, and thus I was not sending a raycast from the center of the dice, and in some scenarios I completely missed the dice altogether.

The code for it is pretty simple though

js
// faceindex to dice face
let faceToDice = {
  3: 1,
  16: 2,
  1: 3,
  18: 4,
  12: 5,
  14: 6,
  2: 7,
  10: 8,
  9: 9,
  6: 10,
  13: 11,
  11: 12,
  8: 13,
  19: 14,
  7: 15,
  5: 16,
  0: 17,
  17: 18,
  4: 19,
  15: 20
}
dice.forEach((d, i) => {
  //Get the position of the dice
  let trans = d.translation()
  //send the raycast from above the dice in a downwards direction
  const raycaster = new Raycaster(new Vector3(trans.x, trans.y + 5, trans.z), new Vector3(0, -1, 0), 0.1, 15)
  let intersects = raycaster.intersectObjects(three.scene.children, true)
  //The raycast could intersect with more than one dice, so filter out our own
  intersects = intersects.filter(intersect => intersect.object.id === diceMesh[i].id)
  let inter = intersects[0] // raycast will likely exit the other side and give 2 faces. ordered by distance first should be pointing up
  console.log(`rolled a ${faceToDice[inter.faceIndex]}`)
})

Ending words

That is basically it. Not an extremely complex task, but it took more time than I imagined.Is this the right way of doing it? no, 100% not. For instance I wouldn’t trust that these dice are perfectly random, even though they should be in theory.

To do this “right” I think it is a better approach to run through the simulation and check what faceIndex is pointing up, and then with a normal Math.random() paint on the value on that face, instead of leaving the randomness up to the physics engine.

Final code

In case you want to reference my code, here it is

diceRoller.svelte
html
<script>
  import { useLoader, DirectionalLight, AmbientLight, PerspectiveCamera, Mesh, useThrelte } from '@threlte/core'
  import { OBJLoader } from 'three/examples/jsm/loaders/OBJLoader'
  import { OrbitControls, T } from '@threlte/core'
  import { World, RigidBody, AutoColliders, Collider } from '@threlte/rapier'
  import {
    MeshStandardMaterial,
    ShadowMaterial,
    CircleGeometry,
    TextureLoader,
    NearestFilter,
    LinearMipMapNearestFilter,
    Raycaster,
    Vector3,
    DoubleSide
  } from 'three'
  import { degToRad } from 'three/src/math/MathUtils'
  import { spring } from 'svelte/motion'
  import { onMount } from 'svelte'
  // faceindex to dice face
  let faceToDice = {
    3: 1,
    16: 2,
    1: 3,
    18: 4,
    12: 5,
    14: 6,
    2: 7,
    10: 8,
    9: 9,
    6: 10,
    13: 11,
    11: 12,
    8: 13,
    19: 14,
    7: 15,
    5: 16,
    0: 17,
    17: 18,
    4: 19,
    15: 20
  }
  const three = useThrelte()
  const scale = spring(1)
  const angularStrength = 10
  /**
   * @type {import("three").BufferGeometry}
   */
  let d20Mesh
  let d20Texture
  const loader = useLoader(OBJLoader, () => new OBJLoader())
  const textureLoader = useLoader(TextureLoader, () => new TextureLoader())
  onMount(() => {
    textureLoader.load('/assets/d20.jpg', texture => {
      texture.magFilter = NearestFilter
      texture.minFilter = LinearMipMapNearestFilter
      d20Texture = texture
    })
    loader.load('/assets/d20.obj', obj => {
      d20Mesh = obj.children[0].geometry
      d20Mesh.computeVertexNormals()
      d20Mesh.normalizeNormals()
      console.log(d20Mesh)
    })
    setTimeout(rollDice, 300)
  })
  function random(min, max) {
    // min and max included
    return Math.random() * (max - min + 1) + min
  }
  /**
   * @type {Array.<import('@dimforge/rapier3d-compat').RigidBody>}
   */
  let dice = new Array(10).fill(undefined)
  /**
   * @type {Array.<import("three").BufferGeometry>}
   */
  let diceMesh = new Array(10).fill(undefined)
  /**
   * @type {import("three").PerspectiveCamera}
   */
  let camera
  async function rollDice() {
    dice.forEach((d, i) => {
      d.setTranslation({ x: random(-1, 1), y: 4, z: 8 }, true)
      d.setLinvel({ x: random(-2, -2), y: 0, z: random(-18, -14) }, true)
      d.setAngvel(
        {
          x: random(-angularStrength, angularStrength),
          y: random(-angularStrength, angularStrength),
          z: random(-angularStrength, angularStrength)
        },
        true
      )
    })
    await waitUntilStopped()
    console.log('dice roll complete')
    dice.forEach((d, i) => {
      let trans = d.translation()
      const raycaster = new Raycaster(new Vector3(trans.x, trans.y + 5, trans.z), new Vector3(0, -1, 0), 0.1, 15)
      let intersects = raycaster.intersectObjects(three.scene.children, true)
      intersects = intersects.filter(intersect => intersect.object.id === diceMesh[i].id)
      if (intersects.length == 2) {
        let inter = intersects[0] // raycast will likely exit the other side and give 2 faces. ordered by distance first should be pointing up
        console.log(`rolled a ${faceToDice[inter.faceIndex]}`)
      } else {
        console.log('broken dice')
        if (intersects.length == 1)
          console.log(`only found one face at ${intersects[0].distance}. Should have gotten an exit hole`)
        else {
          console.log('did not find anything')
        }
      }
    })
  }
  let last
  async function waitUntilStopped() {
    const precision = 4
    const diceIndex = d => ({ trans: d.translation(), rot: d.rotation() })
    if (!last) {
      last = dice.map(diceIndex)
    } else if (
      dice.map(diceIndex).every((dI, i) => {
        return (
          dI.trans.x.toFixed(precision) === last[i].trans.x.toFixed(precision) &&
          dI.trans.y.toFixed(precision) === last[i].trans.y.toFixed(precision) &&
          dI.trans.z.toFixed(precision) === last[i].trans.z.toFixed(precision) &&
          dI.rot.x.toFixed(precision) === last[i].rot.x.toFixed(precision) &&
          dI.rot.y.toFixed(precision) === last[i].rot.y.toFixed(precision) &&
          dI.rot.z.toFixed(precision) === last[i].rot.z.toFixed(precision)
        )
      })
    ) {
      return
    }
    last = dice.map(diceIndex)
    await pause()
    return waitUntilStopped()
  }
  function pause() {
    return new Promise(function (resolve, reject) {
      setTimeout(resolve, 150)
    })
  }
</script>
<PerspectiveCamera bind:camera position={{ x: 0, y: 40, z: 0 }} fov={24}>
  <OrbitControls maxPolarAngle={degToRad(80)} enableZoom={true} target={{ y: 0.5 }} />
</PerspectiveCamera>
<DirectionalLight
  shadow={{
    mapSize: [2048, 2048],
    camera: {
      left: -20,
      right: 20,
      top: 20,
      bottom: -20,
      near: 0.0001,
      far: 100
    },
    radius: 8
  }}
  intensity={0.1}
  target={{ x: 1 }}
  position={{ x: 0, y: 10, z: 10 }} />
<DirectionalLight position={{ x: -3, y: 10, z: -10 }} intensity={0.3} />
<AmbientLight intensity={0.2} />
<World>
  {#if d20Mesh && d20Texture}
    {#each dice as d, i}
      <RigidBody
        bind:rigidBody={d}
        position={{ x: Math.random(), y: 4, z: 8 }}
        linearVelocity={{ x: Math.random() * 8 - 4, y: Math.random(), z: Math.random() * -20 }}
        angularVelocity={{
          x: Math.random() * angularStrength,
          y: Math.random() * angularStrength,
          z: Math.random() * angularStrength
        }}>
        <AutoColliders shape="convexHull" mass={0}>
          <!-- Cube -->
          <T.Group scale={$scale}>
            <Mesh
              bind:mesh={diceMesh[i]}
              position={{ x: 0, y: 0, z: 0 }}
              castShadow
              geometry={d20Mesh}
              material={new MeshStandardMaterial({ map: d20Texture, side: DoubleSide })} />
          </T.Group>
        </AutoColliders>
      </RigidBody>
    {/each}
  {/if}
  <Collider shape="cuboid" args={[100, 0.1, 100]} />
  <Collider shape="cuboid" args={[1000, 100, 1]} position={{ x: 0, y: 0, z: 10 }} />
  <Collider shape="cuboid" args={[1000, 100, 1]} position={{ x: 0, y: 0, z: -7 }} />
  <Collider shape="cuboid" args={[1, 100, 1000]} position={{ x: 10, y: 0, z: 0 }} />
  <Collider shape="cuboid" args={[1, 100, 1000]} position={{ x: -10, y: 0, z: 0 }} />
  <Mesh
    receiveShadow
    interactive
    on:click={rollDice}
    rotation={{ x: degToRad(-90) }}
    material={new ShadowMaterial({ color: '#050505' })}
    geometry={new CircleGeometry(300, 72)} />
</World>

Basics

Lets begin with the most basic example of 4d6. It has 3 components, a number followed by a letter d followed by another number. So lets create a parser for it with nearley

# this is our starting rule
Main -> int "d" int
# We are only interested in integers. floats does not make sense here
int -> [0-9]:+        {% (d) => parseInt(d.flat().join("")) %}

Note that the int rule produces an array of strings, so with the postprocessing function we can make sure that it outputs an integer.

So, parsing 4d6 will output [4, 'd', 6].

Next up, lets add a parser for the optional +1 at the end

Main -> int "d" int
Main -> int "d" int "+" int

And we are pretty much done. But I think we can do better, by adding a few features.To begin with it would be good if we could use more than one type of dice. For instance 2 6 sided dices and 1 10 sided dice 2d6 1d10.

Multiple dice and recursion

To do this we can begin by refactoring the main rule into a dice rule

Main -> Dice
Dice -> int "d" int         {% (d) => ({count: d[0], sides: d[2], bonus: 0}) %}
      | int "d" int "+" int {% (d) => ({count: d[0], sides: d[2], bonus: d[4]}) %}

Note that we are also formatting the output into an object to make it less error prone.Now to add multiple dices it is as easy as adding right hand recursion on the Main rule.

Main -> Dice
      | Dice " " Main {% (d) => [d[0], d[2]].flat()%}

The post processing step on the second rule makes it so that we get an easy to parse output with an array of dice objects, instead of a deeply nested array.

Advantage/Disadvantage

Last thing I want to add to the language is something specific to dungeons and dragons. In that game it is common to throw twenty sided dices with either advantage or disadvantage. This means you throw the dice twice and keep the largest or smallest result respectively.

I think I want to represent this as 1d20a or 1d20d. This got a bit messy, but here are the rules I made for it

Main -> BonusDice
      | BonusDice " " Main {% (d) => [d[0], d[2]].flat()%}
BonusDice -> EnhancedDice　        {% (d) => ({...d[0], bonus: 0}) %}
       | EnhancedDice "+" int  {% (d) => ({...d[0], bonus: d[2]}) %}
EnhancedDice -> Dice      {% (d) => ({...d[0], advantage: false, disadvantage: false}) %}
              | Dice "a"  {% (d) => ({...d[0], advantage: true, disadvantage: false}) %}
              | Dice "b"  {% (d) => ({...d[0], advantage: false, disadvantage: true}) %}
Dice -> int "d" int         {% (d) => ({count: d[0], sides: d[2]}) %}

Polish

The last convenience I want in the language is the option to not specify how many dices to throw, and default to 1. So 1d20 should be the same as d20. Pretty simple to just add to Dice

Dice -> int "d" int         {% (d) => ({count: d[0], sides: d[2]}) %}
      | "d" int             {% (d) => ({count: 1, sides: d[1]}) %}

That is basically it!The output can easily be parsed and you can create a dice roller.Or, you can do like I did and add the dice rolling straight into the parser.

# this is our starting rule
Main -> MultiDice {% (d) => d[0].map(v => ({...v, sum: v.rolls.reduce((acc, val) => acc + val) + v.bonus})) %}
MultiDice -> BonusDice
           | BonusDice " " MultiDice {% (d) => [d[0], d[2]].flat()%}
BonusDice -> RolledDice　        {% (d) => ({...d[0], bonus: 0}) %}
         | RolledDice "+" int  {% (d) => ({...d[0], bonus: d[2]}) %}
#Do the roll directly in the parsing
RolledDice -> EnhancedDice {% (d) => ({...d[0], rolls:
   new Array(d[0].count)
   .fill(undefined)
   .map(() => {
   //Create two rolls for each dice
   const rolls = new Array(2).fill(undefined).map(() => Math.floor(Math.random() * d[0].sides + 1))
   if(d[0].advantage)
   return Math.max(rolls[0], rolls[1])
   if(d[0].disadvantage)
   return Math.min(rolls[0], rolls[1])
   else return rolls[0]
   })}) %}
EnhancedDice -> Dice      {% (d) => ({...d[0], advantage: false, disadvantage: false}) %}
              | Dice "a"  {% (d) => ({...d[0], advantage: true, disadvantage: false}) %}
  | Dice "b"  {% (d) => ({...d[0], advantage: false, disadvantage: true}) %}
Dice -> int "d" int         {% (d) => ({count: d[0], sides: d[2]}) %}
      | "d" int             {% (d) => ({count: 1, sides: d[1]}) %}
int -> [0-9]:+        {% (d) => parseInt(d.flat().join("")) %}

It got a bit messy, but pretty neat to get an output with a list of all the dice values and the result of adding them all together

Here is an example of the output if run with the input 1d20a+2 d4+1

json
[
  {
    count: 1,
    sides: 20,
    advantage: true,
    disadvantage: false,
    rolls: [14],
    bonus: 2,
    sum: 16
  },
  {
    count: 1,
    sides: 4,
    advantage: false,
    disadvantage: false,
    rolls: [4],
    bonus: 1,
    sum: 5
  }
]

Final words

Nearley might not have been the best fit for this, but it was in all honesty really fun to use. And the result speaks for itself, it works and is pretty neat.The best part is that it is quite easy to build upon.

Now all that is missing is to starting using this system, and while implementing this I was thinking of creating a system like the one in DnDBeyond where you have 3D dice rolling on the screen.Would be fun to experiment with, and should be doable using three.js without too much effort

For my upcoming Dungeons and Dragons I wanted to do all of my world building in a computer readable format.And I was pretty sure I wanted to write that format as well, so I picked yaml.One could ask why I didn’t pick ex markdown and it is because the structure wouldn’t feel right for me, and the metadata does not work the way I want it to.For example I think it would be a good idea to have one single yaml file for the whole of a political organization in my game.That one yaml file can then easily contain the history of the organization, all needed NPCs, and potential quest lines, something I don’t find viable in a markdown format.

Documentation entry

Here is an example of a yaml file I created for my D&D world (in a cyberpunk setting)

yaml
name: Neon Chroma
description: The neon chroma is a really small bar with all seating out on the road. It is dimly lit with neon signs
npc:
  - name: Neo
    race: near human
    description: A charismatic android with a dry wit, Neo enjoys chatting with patrons about the latest tech advancements.
signatureDrink: |-
  Neo's Blissful Elixir: Create a unique blend of equal parts vodka, amaretto, and blueberry flavored syrup. Pour into an ice-filled shaker and shake vigorously. Serve in a martini glass with a lime wedge garnish.

Backend

I think this is a pretty nifty way of documenting, but there is one issue.Across the many files created, how do I find all the signature drinks that exists in my world?… ok, not a likely query, or thought to have, but you get the idea.

I suppose I could push it into a real indexing engine, such as Elasticsearch, but wow that would be such an overkill for a some homebrew game.Another less extreme option would be to use a normal NOSQL database such as mongo, which would be able to query this data for me.But, I would prefer to skip the complexity of having a separate server/database all together.

This led me to implement my own simple query system.

Querying

js
query(`.signatureDrink`)

That query should be able to fetch all drinks for me.So how to implement it?

I did a naive approach of reading all yaml files into memory, which honestly works fine, my campaign really isn’t that big, and is unlikely to ever get so big that it becomes an issue.

I then wrote a small script

typescript
function nextQry(query) {
  if (query.substring(1).split("").includes(".")) {
    return query.substring(1).split(".").slice(1).join(".");
  }
}
function arrayQuery(array, query) {
  if (query.startsWith("[")) {
    console.log("array search");
    const item = eval("array" + query.substring(0, query.indexOf("]") + 1));
    if (item && typeof item === "object") return anyQuery(item, nextQry(query));
    if (item) return item;
    return;
  }
  if (!query.startsWith(".")) {
    query = "." + query;
  }
  var newArray = [];
  for (var i = 0; i < array.length; i++) {
    var item = array[i];
    if (typeof item === "object") {
      const res = anyQuery(item, query);
      if (res) {
        newArray.push(res);
      }
    } else if (eval("item" + query)) {
      newArray.push(item);
    }
  }
  return newArray;
}
function objectQuery(obj, query) {
  if (!query.startsWith(".")) {
    query = "." + query;
  }
  const next = nextQry(query);
  const current = eval("obj" + query.split(".").slice(0, 2).join("."));
  if (next && current) return anyQuery(current, next);
  if (current) {
    if (typeof current === "number" || typeof current === "string")
      return current;
    return obj;
  }
}
function anyQuery(anyData, query) {
  if (Array.isArray(anyData)) return arrayQuery(anyData, query);
  return objectQuery(anyData, query);
}

And with that I could send an array of all my files (parsed from yaml) into anyQuery and get a list of all drinks in all files.

Of course this assumes that I spell the property correctly every time, but I suppose I can’t really get around human errors

Prototypes

I started making a bunch of prototypes with a lot of confidence, and little knowledge of the domain.I remember one of my early prototypes where I got annoyed that a character could mean more than one thing and be pronounced in more than one way… Absolute madness, and obviously “they” made a mistake when creating the japanese language.

Anyways the early websites I made are not really the topic for this post, no matter how polished or unpolished they were. Instead I want to focus on the latest webapp I made.

Wishlist

I had the idea to create a learning app with vocabulary lessons, grammar lessons, and story driven lessons where you try and have a scripted somewhat realistic chat conversation.And to top it all up, creating the lessons (or content) to the app had to be easy and not hindered by annoying repetitive formats such as json or yaml. Seems like an easy task right?

Compiler

So I did what ever other sane language learner with a bit of programming know how would do… I wrote my own compiler for a custom DSL.

Now, don’t get me wrong: making a compiler is a massive overkill and can be a daunting task, but I had scoped it to such a level where it really wasn’t that big of a task.But anyways, I started out the way I always do, which can only be described as the wrong way. I started writing code first without doing the slightest bit of research on how compilers generally works, because how hard could it be?

Well turns out it is at least a bit tricky and my first iteration ended up as a pile of spaghetti code that I barely worked and was so rigid that I could forget about adding any features that I wanted.

So I scrapped it, and started doing a little bit of research and stumbled upon nearley which I believe would be absolutely great for most compiler projects. Unfortunately it wasn’t really possible for me to use it because I could for the life of me figure out how to make it tab/space aware, and I wanted my language to have semantically important spaces the same way yaml does.

Back to square one and I decided to make everything myself… but which some background knowledge.So I made it the way I should have the first time, with a tokenizer and parser which gave me an output in the form of a parse tree.

I could then take that output and compile it into some svelte code, which worked great… unfortunately I completely burned out on the project before it got even remotely useful.

Here is an example of the input to the compiler

vocabulary:
  もしもし
  hello
  先[せん]週[しゅう]
  last week
grammar:
  title: い adjective conjugation
  conjugating in affirmative inflections
  present | past
  [い]です | [-い-]かったです
  ex:
    present  | past
    たのしいです | たのしかったです
    おいしいです | {text hiragana | validate おいしかったです}
    おおきいです | {text hiragana | validate おおきかったです}
story:
  > 先[せん]週[しゅう]仕[し]事[ごと]はどうでしたか
    how was last weeks work?
  < 忙[いそが]しかったです
    {text hiragana | validate いそがしかったです}
                   | suffix いです | hint "You are not conjugating to past tense"}
    {help "base for of busy is 忙[いそが]しい. To make any い adjective into past affirmative tense you remove the trailing い and add かった"}
    It was busy

oh well, for the next project at least I know how to write a compiler.