From 26C3 Public Wiki

Jump to: navigation, search

Dyncall

File:WELIKEPORN-630px.png

The Dyncall Project

Dyncall is a minimalistic middleware software project released under two-clause BSD license. It has been started as an alternative approach to the major problem of making binary components such as shared libraries and dynamic programming languages talk with each other without extra glue code.

Contents

Architecture

Small call kernels written in Assembly language for different hardware platforms have been written that implement multiple calling conventions on the machine-level. A portable interface has been designed in C. Several bindings for programming languages have been implemented.

Recently, callbacks have been added to the library. An exhaustive test suite rounds up the library that can test extreme situations such as function calls and callbacks with dozens to hundreds of arguments out of the complete set of C types.

Foreign Function Call - C Interface

Calling a foreign function with dyncall is done through a call vm which has to be created once with

DCCallVM* pCallVM =  dcNewCallVM(4096 /* bytes ... */ );

The call invocations from the call vm take place on the C stack ; the 'size' argument specifies the maximum size provided to buffer all arguments for exactly one call, so 4096 is more then enough for almost all run-time scenarios. load-and-copy argument buffer Then three steps are needed to specify a foreign functions.

  1. select target calling convention (optional, defaults to standard C on platforms)
  2. reset internal data vector
  3. preload arguments step-by-step
  4. invoke by specifing target address and return type

The following will give a short example which calls the libSDL library. Given, libSDL is already loaded and we resolved the following C function symbol at run-time:

SDL_Surface* SDL_SetVideoMode(int w, int h, int bpp, int flags);

We want to call this function with w = 640, h = 480, bpp =32, flags = SDL_OPENGL|SDL_DOUBLEBUF and return type is 'pointer'. So we first specify the calling convention:

dcMode(pCallVM, DC_CALL_C_DEFAULT);

more complicated settings are possible such as 'DC_CALL_C_X86_WIN32_STD' to call windows system DLLs such as OpenGL 1.1 from the Win32 API. Next we push arguments from left to right according to the C prototyp:

/* interface: 'dcArgXXX()' functions where XXX is 'argument type' */
dcArgInt(pCallVM, 640);
dcArgInt(pCallVM, 480);
dcArgInt(pCallVM, 32);
dcArgInt(pCallVM, SDL_OPENGL|SDL_DOUBLEBUF);

Finally, we issue the foreign function call by specifying the return type and target address.

/* interface: 'dcCallXXX()' functions where XXX is 'return type' */
void* ptr = dcCallPointer(pCallVM, address /* here = &SDL_SetVideoMode */ );

Call and Callback Signatures: high-level portable interface

The same can be specified in languages such as R using the dyncall bindings and a function call signature string:

R:

surface <- .dyncall( address, "iiii)p", 640, 480, 32, SDL_OPENGL + SDL_DOUBLEBUF )

The format is simply

argument-types ... ')' result-type

specified from left-to-right according to the C prototyp.

Here's a table of signature encodings and corresponding C/C++ types...

'B' Bool
'c' char
'C' unsigned char
's' short
'S' unsigned short
'i' integer
'I' unsigned integer
'j' long
'J' unsigned long
'l' long long, int64_t
'L' unsigned long long, uint64_t
'f' float
'd' double
'p' pointer
'Z' C String
'*...' pointer of .... (high-level)
'<'...'>' struct/union tyoe .... (high-level)

In addition, we are brainstorming these days about a common calling-convention prefix notation.

In the R binding we get one step further to support also structures.

... to be continued...

Dynports

On top of these libraries, we are currently experimenting with "dynports" - cross-platform portable dynamic bindings of 'whole C libraries' using a C header parser tools (gccxml,xslt,boost::wave) to gather meta-information. These meta-information plus dyncall can be used to implement a generic binding into dynamic programming languages. A first prototype is available in the R language bindings of dyncall.

Scripting Multimedia and OpenGL Shaders in R

Tests include more complicated settings such as OpenGL 3 bindings using libSDL, glew, OpenGL on multiple platforms (Windows, Linux, Mac OS X,FreeBSD) within the R language

I (pl3x) will be at 26C3 and will have live demonstrations on multiple platforms with me. So if you are interested - please come up!

TODO and Hacking at 26C3

Here is a list of things to explore, contribute and play with -- maybe someone is interested in that topic:

"OS" foreign function calls and test-suites

Dyncall provides the ability right now to make OS foreign function calls on many architecture ports (uptodate: x86 and ppc32). What's missing currently is a test-suite that e.g. loads existing or prepared OS-foreign binary containers (e.g. ELF on windows/darwin, PE on Linux/BSD/mac os x, Mach-O on windows/linux/bsd...) If someone is experienced with those stuff - we'll be glad to hear from you. This could also contribute to the dynload library - part of dyncall.

'clang' based C/C++ header parsers

clang (http://clang.llvm.org/) is parser front-end for C/C++ -- part of the LLVM project (http://llvm.org).

Tune-up boost::wave based pre-processor to evaluate complex macro bodies

Currently 'bw' - the boost::wave based C preprocessor parser of dynports is not able to evaluate complex C Macro bodies such as implicit C cast and arithmetic expressions. So what is needed is a macro evaluation phase.

New Ports

The following architectures have not been implemented: sparc, sparc64, ppc64, itanium, 68k, 88k, 6502, mips64, alpha, sh, cell, blackfin, z80, hppa, vax In addition - many calling conventions are still missing (borland, watcom, intel?) - but some of those are already documented in the manual with stack layout descriptions. So, if you feel like want to write some "useful" assembly on your beloved processor architecture - we really love to talk with you!

Contact

Keywords: middleware, language bindings, assembly, foreign function calls, callbacks, thunks, dynamic bindings, component model

Website: http://dyncall.org

Developer contact: pl3x

26C3 Public Wiki

MistyLook for WordPress originally by Sadish Bala