13
$\begingroup$

I hear a lot about Q/kdb+. I've never had the opportunity to use it for anything real but have played with it using their trial license and found it intriguing (if not somewhat mind warping). I've seen a few references to the J language which is also a derivative of APL according to Wikipedia and have heard of at least one quant trader using it as a poor man's replacement for Q/kdb+.

Has anyone put J to use in the real world? Is it worth investing time and energy into to learn, especially if I'm not interested in investing the dollars required to get a kdb+ license? I suspect a lot of K/Q's value comes from the kdb+ database that underlies it. I'm not sure there is an equivalent for J, but I haven't performed an exhaustive search.

Interestingly enough, J recently went fully open source and is now hosted (not officially) on Github: https://github.com/openj/core.

EDIT: Assuming some folks here are familiar with J and Q/kdb+ but have not used J extensively in industry, what might be some major advantages or disadvantages of using J.

EDIT 2: J has a database solution in JDB. One obvious follow-up which I also asked in a comment to @chrisaycock's answer below is how JDB might compare to kdb+ in terms of functionality.

Improve this question
$\endgroup$
3
  • $\begingroup$ That GitHub repo is from a fan, not an employee of J Software. The official source is here. And yeah, I've actually thought about cloning kdb+ for J precisely for the open-source crowd, though I'd have to get a boat-load of time before getting around to that. $\endgroup$
    – chrisaycock
    Commented Sep 9, 2011 at 1:49
  • $\begingroup$ Thanks for the correction, I've updated the question to indicate that the Github is unofficial. $\endgroup$
    – Louis Marascio
    Commented Sep 9, 2011 at 1:55
  • $\begingroup$ Maybe rather than cloning kdb+ totally it would be worth implementing J on an opensource datastore. I remember looking at this previously and coming to the conclusion that something like tokyo cabinet might fit the bill. That said I may be getting this totally wrong as I have had NO exposure to q/kdb+ and relied on what I could find online. (Yes you can get a trial version of kdb but my mind had wandered by the time I discovered the fact) $\endgroup$
    – Jonathan Evans
    Commented Mar 24, 2012 at 3:19

3 Answers 3

Reset to default
7
$\begingroup$

You are right that the database component is the main selling point of q/kdb+. Indeed, many Kx customers use kdb+ just for time-series data (and often just for tick history) and don't take advantage of the q language. So the first disadvantage to using J as-is would be to miss the reason most folks even bother with Kx: data storage.

Now of course you could clone the data storage for J, just as you could for any language. The really hard part is coming-up with your own version of the query and join routines found in q.k, that cryptic file that comes with the q software distribution.

Regarding JDB, the biggest difference I can find is that it doesn't support partitioning. For example, kdb+ can split the column-mapped files into separate directories for each date. That means a query select from quotes where date=2011.09.08 will cause kdb+ to jump directly to today's ticks. Taking that further, kdb+ supports threading to some degree (via par.txt) so that it can process multiple dates in parallel. And beyond the performance benefits, partitioning is practically required for really large tables anyway since the columns are memory-mapped at query time; there's no feasible way to do that for a multi-terabyte database without some selectivity.

The second disadvantage is that there aren't nearly as many finance shops that use J. Almost every big bank has a Kx license somewhere, so there's always an expert if needed. J doesn't have nearly that adoption.

Of course, J's main advantage now is that it's open source. And even before going open, J Software was more academic-friendly than Kx Systems, so there's more presence of J in the universities. Because of that, there seems to be more J examples in papers and on Project Euler than for either k or q.

Improve this answer
$\endgroup$
2
  • $\begingroup$ How does kdb+ compare to J's JDB? This might also be a separate question all together I suppose. $\endgroup$
    – Louis Marascio
    Commented Sep 9, 2011 at 2:26
  • $\begingroup$ I've updated my answer. $\endgroup$
    – chrisaycock
    Commented Sep 9, 2011 at 2:48
1
$\begingroup$

to add to chrisaycock's reply,

J has Jd database (comercial) now. Its a purer vector language than q, but can be used with the same simplicity.

Improve this answer
$\endgroup$
-1
$\begingroup$

If you're looking for an alternative to Q, try R. It's a a great language for modeling and time series analysis. For a DB, try cassandra, which supports partitioned data, high-speed access/queries.

Nice thing is that you can quickly produce compelling graphs and charts in R.

Also, R has a big community compared to J. And it's all open source, with a strong, dynamic base of users...so you won't get stuck with code that no one can support or understand.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.