music notation through JavaScript

I’m working on a project which will hopefully make me rich (mwuahahaha!).

The idea is that I build an application, where people with MIDI keyboards can test their sight-reading online.

The program will judge your playing objectively, according to whether you hit the right notes, what your timing was like, whether you got the dynamics right, etc.

But anyway – in order to do that, some technical hurdles needed to be cleared:

  • 1. how do you read from a MIDI keyboard in JavaScript?
  • 2. how do you render notation online in JavaScript?

Before you ask – I’m using JavaScript because that’s what I’m good at. JavaScript for the front end, and PHP for the back.

The first part already has a solution – not too long ago, Daniel created a Java applet which reads from the keyboard and passes the key events to Flash, by way of JavaScript.

So all I need to do there is to hijack the middle part, which is not difficult.

On the rendering side, though, there does not appear to be any existing music notation rendering software for JavaScript.

One solution might be to pre-render the notation with a console app and save them as images. That’s not perfect, though, because I want to be able to highlight wrong notes, and render the sheet in different ways according to how the reader’s screen is set up.

So – I need to write a renderer in JavaScript.

Here’s the beginning of it (source).

notation-screenie

To work on this, I will be going through my piano books and picking out pieces that are gradually more difficult for the notation renderer to try to render. Soon enough, it will be good enough for use.

Its current features:

  • reads from compressed MusicXML files.
  • renders crochets and minims, with appropriate padding between all notation symbols.
  • will wrap onto a second line of staff (or further if need be) if there is not enough room in just one.

Short-term upcoming features:

  • render two staves (bass and treble) to show left and right hands.
  • render semi-breves</li
  • show fingering on notes
  • dotted notes
  • quavers
  • tied quavers

The big idea with the application is that the site will be able to gradually increase the difficulty of the pieces, according to how well the player performs.

It’s not confined to sight-reading either – there’s no reason why this idea could not also be used to teach full songs.

jQuery 1.3 With PHP: calendars chapter is online

Packt have put one of the chapters of my book, jQuery 1.3 with PHP, online as an article.

I mentioned this chapter in June, and provided a small demo and the source.

I’ve had a few requests for example code on how to do this using a real database. I’ve been meaning to make the time to do that. I’ll get that done hopefully tomorrow.

In the meantime, though, please enjoy the article.

By the way, the book went to the printer only a few days ago, so if you want to avail of the pre-order price, now’s the time.

kfm2: example widget

As an example of a front-end widget, I’ve whipped up a demo of file-tree navigation using the excellent jsTree widget.

here is the demo

You can see from the source how short it is – all it does is load up the jsTree library (and jQuery), then it connects directly to KFM2 to get data about the files on the server.

in fact… it’s so short I’ll just paste the demo into this page as well:

$(document).ready(function () {
$(“#tree_div”).tree({
data : {
type : “json”,
async : true,
opts : {
async : true,
method : “POST”,
url : “http://demo.verens.com/kfm2/trunk/?p=get_file_listing” // KFM LOCATION
}
},
callback : {
‘beforedata’:function(node){
var id=$(node).attr(‘id’);
if(!id)return {‘d’:’/’};
return {‘d’:$(node).attr(‘id’).replace(/^kfm2_node_/,”)};
},
‘ondata’:function(res){
var data=[],i;
for(i in res.files)data.push({‘data’:i,’attributes’:{‘id’:’kfm2_node_’+res.d+’/’+i}});
for(i in res.directories)data.push({‘data’:i,’state’:’closed’,’attributes’:{‘id’:’kfm2_node_’+res.d+’/’+i}});
return data;
}
}
});
});

You should see a list of files just above this paragraph.

No modification of jsTree was necessary, as it handily gives two event triggers which can be used to translate the request data to KFM’s request format, and the received data back into jsTree’s data format.

Here is the code used to attach the tree to the #tree_div element in this page:

$("#tree_div").tree({
	data : {
		type : "json",
		async : true,
		opts : {
			async : true,
			method : "POST",
			url : "http://demo.verens.com/kfm2/trunk/?p=get_file_listing" // KFM LOCATION
		}
	},
	callback : {
		'beforedata':function(node){
			var id=$(node).attr('id');
			if(!id)return {'d':'/'};
			return {'d':$(node).attr('id').replace(/^kfm2_node_/,'')};
		},
		'ondata':function(res){
			var data=[],i;
			for(i in res.files)data.push({'data':i,'attributes':{'id':'kfm2_node_'+res.d+'/'+i}});
			for(i in res.directories)data.push({'data':i,'state':'closed','attributes':{'id':'kfm2_node_'+res.d+'/'+i}});
			return data;
		}
	}
});

This is just an example of what can be built quickly with the new RPC-based KFM.

Further and more useful examples will come soon.

kfm2: plugin management

I know, I promised this last week – I’m afraid I’ve been up to my eyes in work and just exhausted at home, so all I can do is apologise.

Anyway, I’ve put together the basics enough that there is one single plugin, which gets a directory listing, and echoes it to the screen in JSON.

You can get a copy of the basics using the following SVN command:

svn checkout http://kfm.googlecode.com/svn/kfm2/ kfm2

In that, there are two directories – trunk and plugins. The trunk is what will become the new KFM core, and the items in plugins will be server-side plugins that can be used by copying them into trunk/plugins/ (or do as I do and ln -s ../../plugins/* from the trunk/plugins/ to create symlinks).

When you load up KFM2 initially, it will do an extremely basic install, with no plugins enabled. Follow the instructions on-screen to get set up.

The config.php contains just two items at the moment. I want to keep this as absolutely clean as possible, so the items in there are the only essentials – all other configuration will be done using KFM’s admin section.

  • KFM_USERDIR – this is the directory that is to be managed by KFM. Just like KFM1, a directory called .files will be created in there that holds config stuff, such as the list of enabled plugins, etc.
  • KFM_ADMIN_PASSWORD – this password allows access to KFM’s admin section, where you can currently just enable/disable plugins. By default, it is blank, and you will need to change it before you can access the admin section.

There is no JavaScript yet in this, so it’s tricky to see exactly what it’s supposed to do.

I’ve set up a basic demo here, but loading that directly will just get you a login screen.

KFM2 is designed to be interacted with purely through RPC (except for configuration which requires browser access). The demo has the first plugin, “get_file_listing” enabled, which we can use to see what happens.

/?p=get_file_listing – this example calls KFM2 and tells it to run the get_file_listing plugin. by default, it will return the root directory of your KFM_USERDIR in JSON format:

{
  "d":"",
  "files":{
    "file2":{
      "size":0,
      "mtime":1255875795
    },
    "file1":{
      "size":0,
      "mtime":1255875794
    },
    "file3":{
      "size":0,
      "mtime":1255875796
    }
  },
  "directories":{
    "dir1":{
      "mtime":1255875809
    }
  }
}

The d parameter there is the directory that was requested (by default, it’s blank), which is internally added to the KFM_USERDIR to get the actual local directory.

The results returned are separated into two arrays, of files and directories. My guess is that most required uses on the client-side will be specific to either directories or files, so this saves you having to separate them yourself.

Changing the requested directory is simple – just add a d parameter to the request URL:

/?p=get_file_listing – this example requests the contents of /dir1.

How to write your own plugin

The system as it is will let you write plugins that handle most of the things that you might want to do – upload files, create/move/delete files or directories, download files, etc.

I’ll get around to writing these anyway, but if you feel like writing them, then feel free! Just copy how the get_file_listing plugin works.

So, here’s how a basic plugin works.

In KFM2’s plugin’s directory, create a directory named after the plugin. Please use just lowercase letters and underscores. Examples: “upload”, “get_file”, “rename_file”, “zip_directory”, etc.

In that directory, create a plugin.php file, which describes the plugin:

<?php
$plugin=array(
  'name'=>'Get File Listing',
  'description'=>'Retrieves a list of files and directories within a given directory',
  'version'=>0
);

Use an integer for the version number. It will probably never need to be used, but if it is, it is easier to compare two integers than two of the usual x.y.z version numbers.

When you log into your admin area, you’ll see the details above shown in your plugin management module.

For most plugin cases, you will also need a file named after the plugin itself. Until the events code is written, this is all cases.

In our example, we create a file named plugins/get_file_listing/get_file_listing.php. It should contain a function also named after the plugin:

function get_file_listing(){
  // place plugin code here
}

In the URL above, I had /?p=get_file_listing – what KFM then did was to load up the get_file_listing/get_file_listing.php file and run get_file_listing(). That function then used whatever other things were mentioned in the $_REQUEST array to carry out its work.

That’s pretty much it for now. That should be enough to create most basic plugins.

For the next KFM article, I’ll write up another server-side plugin, and will add in some client-side code so you can see how it would actually be used in a real project. It will demonstrate how JavaScript and KFM will talk to each other.

If anyone writes up any plugins that they want included in the SVN download, please email them to me (kae@verens.com).

Requested plugins

Here’s the list of plugins that have already been identified as needing to be written.

Server-side plugins that can be written right now:

  • file upload
  • delete file / directory
  • rename file / directory (also counts as “move”)
  • download file
  • create zip of directory
  • unzip a zip file
  • manipulate an online image

Plugins that require more work in the core:

  • connect to remote file-systems (FTP, NFS, other KFMs, etc)
  • user-based security
  • different KFM_USERDIR based on the logged-in user, or even the site’s hostname
  • plugin to resize images that were uploaded so they’re kept to a certain max size
  • quotas (can /probably/ be done at the moment… but not sure)
  • databases

KFM2 has begun. requesting ideas of sample plugins to develop

As ye all know now, KFM1.x is finished. I’m not interested in struggling with it anymore. I’ve started a new project which will exceed KFM1.x’s capabilities.

The main problem with 1.x was that it was monolithic – there was one single codebase, which encompassed both the server-side and the client-side. The client-side code was useless without the server-side, and the server-side code had hooks specifically designed to be used by the client-side.

While it was possible to build a different client-side GUI that would hook into the server-side, it would be a large undertaking, and would involve changing a lot of the PHP – especially as the PHP wrote the “boot” JavaScript in the first place…

KFM2 is designed from the beginning to have a different architecture based specifically on plugins. The core code is a small plugin manager (mostly written already), written in PHP, but it will not be necessary that the client-side is in JavaScript – or even that there is any interaction with a browser at all!

The idea is this – if you want to do something incredibly simple, such as select an existing directory that will be used in your CMS to create an image gallery out-of (let’s say you need to choose from a number of existing directories full of images), then it is a total waste of bandwidth and CPU to load up a full-on file-manager.

In this case, and it’s the first I’ll be building to show how all this works, what should happen is that the browser will show something simple like this:

Choose the image directory

…and when that’s clicked, a small popup will appear which lets you choose the directory you want, in a tree-like manner – you open the branches until you find the one you want, then you click it. Afterwards, the form will change:

Choose the image directory /images/the-lads-2009/ selected

From the technical point of view, there’re a few things to explain about how this would be done.

jQuery would be used to attach an event to the button which would popup the selection window. This selection window will use Ajax to speak with the KFM2 server and retrieve the list of directories, one level at a time as the branches are expanded. When one is selected, a hidden input will be set to the name of the directory.

This sounds incredibly simple, and it is. What’s important, though, is that no-one is doing this at the moment!. Right now, if you go search for existing plugins to do this kind of thing, you’ll find either small widgets with both the server and client-side code ready for you, or full-on applications (such as KFM1.x) which can do the job, but which are overkill.

The problem with the applications is obvious – too much stuff going on, making it slow and cumbersome.

The problem with the small widgets is that each one is complete in itself, and therefore a lot of the functionality is repeated. This is a bad idea – what if you need to make a site-wide change? You’d need to go through every widget and change it!

What I’m proposing to make of KFM2 is that it will mostly be a server-side application, but which has a nice easy-to-use API which can be hooked into by lots of different jQuery widgets (or Mootools, or whatever you want, really – write it in Python or Delphi if you want…).

So, my first widget will be the one described above – a directory selector.

I’d like to know what other things people need to do with files online, which are simple and they’d prefer not to have to load a whole file-manager to do.

I’ll release the first widget and a demo of it some time over the next week. I intend to write a whole load of widgets, culminating in a full-on GUI similar to the present KFM1.x one, all hooking into the new engine.

KFM 1.4.2 released. now with CKeditor support

kfm

KFM 1.4.2 has been released. It’s a minor maintenance release, but it adds support for CKeditor. TinyMCE and FCKeditor have been checked as well.

demo using CKeditor

This release also adds a configurable restriction, where images larger than a certain resolution will simply not be allowed in the repository.

Very large images can cause servers to crash when the system tries to generate thumbnails, because of the large amount of RAM needed. For example, a 3000×2000 image will use up at least 24MB just to hold the thing in memory, and much more to handle the manipulation. 24MB is a large amount of RAM to a web app.

There are also some code additions which are not used by the KFM GUI, but can be used by tools that hook into the KFM engine. These allow uploaded files to be placed in directories specified at the time of upload, or to replace existing files, or even to take an uploaded image and apply HSL transformations to it (when ImageMagick is installed). These are not documented, but anyone that needs this functionality will be a good enough programmer to read the source and find it (or email me 😉 ).

Download KFM 1.4.2 here