Posts
-
Reinforcement learning, non-Markovian environments, and memory
I recently finished reading a reinforcement learning textbook (by Barto and Sutton), and throughout it I was constantly vexed by one important assumption that underlined all the algorithms: the Markov property, stating that a succeeding event depends only on the one preceding it, and not on ones far in the past. This kind of restriction is a helpful tool both practically and theoretically, but it causes problems when dealing with a rather natural situation where there is long-term dependence between events. The book doesn’t address this, so I tried to, and found out it’s a genuine problem with real research behind it. Here’s what I did.
-
A game in a pure language (part 2): state
My first post about this project was a general introduction and an overview of my impressions of Idris. In this one, I’ll go over some elementary facilities that enable creating stateful systems like games in a pure language. Also I finally published the game’s source code: github.com/corazza/game-idris (it’s last been tested in Idris 1.3.1).
-
A game in a pure language (part 1): introduction and problems with Idris
At the beginning of 2019, I started working on my most ambitious project yet: a video game in Idris, a pure functional language with dependent types. I picked Idris for two reasons: (1) because I wanted to learn the language as it seemed amazing, and (2) because I was sure that making a game in a pure language was bound to present interesting problems. So far it’s been an amazing journey (albeit frustrating at times): somewhere around 13k lines of Idris code, along with trace amounts of C and C++ needed for some bindings; a number of somewhat-working game systems, one complete rewrite…
-
Petnica International and optimizing glass shapes
Over the last 10 days I've been at Petnica International, a science seminar in Serbia, and it was an incredible experience (not unlike other [Petnica](http://petnica.rs/) programs) - I've met lots of cool and interesting people from all over Europe, learned some new physics and computer science - but this post is mainly about my project there. -
Edge detection using the Sobel operator
In image processing, a lot of useful information in an image is contained in the edges of objects, and edge detection is a process of separating the object edges and the rest of the image (objects themselves and their environments). Obtained information can then be used for further analysis, such as detection of faces, boundaries, or other image processing procedures such as superresolution (e.g. deciding whether to continue a gradient or place a sharper border between new pixels could depend on whether an edge is being processed).
-
Robot Evolution
Robot Evolution is an application that uses genetic algorithms to evolve and optimize virtual walking poly-pedal robots. The robots are 2D geometric constructions of rectangles that are connected by virtual motors which apply torque to these rectangles, making them move.
-
Manipulating Java class loading mechanisms
At the moment I am working as an intern in CROZ, and I was tasked with creating an evaluator for Java for an online competition. It’s essentially supposed to execute code sent by the competitors using JUnit (and a security framework, of course), and return some results - the final implementation is supposed to be a web application which would allow users to submit JAR files with their classes to the server and display a ranking and/or their individual points and test results.
-
Introduction to HORIZONS
While working on Graviton, I discovered a useful system set up by NASA, that provides lots of information about celestial bodies, based on very accurate data (I believe it is also used to guide spaceships). It’s called HORIZONS, and is maintained by NASA’s Jet Propulsion Laboratory.