Showing posts with label Open Source. Show all posts
Showing posts with label Open Source. Show all posts

Thursday, September 15, 2011

Compiling Your Own Version of SQLite for iOS and applying some fixes

Adding your own version of sqlite for iOS is fairly easy task.
  1. Go to http://www.sqlite.org/ and download the latest sqlite amalgamation archive
  2. Extract it somewhere and add it to your Xcode project. You have several options here. You could copy it to your project folder, or link to it. You could also decide, to put it as a static library target and link it to your main target. It's up to you. You will need only sqlite.c and sqlite.h files.
  3. Apply some #defines you might need. For example:
#define SQLITE_ENABLE_FTS3
#define SQLITE_DISABLE_LFS
#define SQLITE_THREADSAFE 0
Then you can compile!

If you need more detail about this part of the process, you can find more detailed description here and here.

And then I hit a problem. I was testing on different simulator versions and found out that for some reason SQLite functions returned strange errors ( SQLITE_CORRUPT with number code 11 ) with some versions of the simulator, while with others, everything worked perfectly fine. After hours of debugging, I've found that the problem is in wrong file size, returned by a function called unixFileSize. Here is how it looks in my sqlite version:
/*
** Determine the current size of a file in bytes
*/
static int unixFileSize(sqlite3_file *id, i64 *pSize){
  int rc;
  struct stat buf;
  assert( id );
  rc = osFstat(((unixFile*)id)->h, &buf);
  SimulateIOError( rc=1 );
  if( rc!=0 ){
    ((unixFile*)id)->lastErrno = errno;
    return SQLITE_IOERR_FSTAT;
  }
  *pSize = buf.st_size;

  /* When opening a zero-size database, the findInodeInfo() procedure
  ** writes a single byte into that file in order to work around a bug
  ** in the OS-X msdos filesystem.  In order to avoid problems with upper
  ** layers, we need to report this file size as zero even though it is
  ** really 1.   Ticket #3260.
  */
  if( *pSize==1 ) *pSize = 0;


  return SQLITE_OK;
}
Here after a successful call to osFstat, buf.st_size contained wrong file size. This is clearly not a bug in SQLite. There might be several reasons for this to happen. Probably struct stat is different in terms of packaging or field sizes from the one compiled with some of the simulators. I don't know and since I cannot fix it in iOS Simulator (the power of closed source in action), I don't really care. What I can fix, is not to use fstat() and struct stat in this function. Here is how I changed the function using other Unix ways and do the job:
/*
** Determine the current size of a file in bytes
*/
static int unixFileSize(sqlite3_file *id, i64 *pSize){
  off_t pos, size;
  pos = lseek(((unixFile*)id)->h, 0L, SEEK_CUR);
  if (-1 == pos){
    ((unixFile*)id)->lastErrno = errno;
    return SQLITE_IOERR_SEEK;
  }
  size = lseek(((unixFile*)id)->h, 0L, SEEK_END);
  if (-1 == size) {
    ((unixFile*)id)->lastErrno = errno;
    return SQLITE_IOERR_SEEK;
  }
  if (-1 == lseek(((unixFile*)id)->h, pos, SEEK_SET)) {
    ((unixFile*)id)->lastErrno = errno;
    return SQLITE_IOERR_SEEK;
  }
  *pSize = size;

  /* When opening a zero-size database, the findInodeInfo() procedure
  ** writes a single byte into that file in order to work around a bug
  ** in the OS-X msdos filesystem.  In order to avoid problems with upper
  ** layers, we need to report this file size as zero even though it is
  ** really 1.   Ticket #3260.
  */
  if( *pSize==1 ) *pSize = 0;


  return SQLITE_OK;
}
As you can see I'm using lseek to get the file size and thus avoid the struct stat and fstat() usage. This fixed the problem for me and I hope this could spare some time to others too. Note that this fix is not portable and will not work in non Unix environment.

Sunday, October 10, 2010

Stages of adopting open source mind-set


These are stages I observed for years from both people and organizations. I might miss something here, but before you think "this is not me/us" think again. I'll use person's perspective, because it always comes to a person. Be it your friend or some big corporate CTO or architect.

  1. Ignorance: open source doesn't exist. If anyone mentions it, you tell yourself "there are dragons there", "it is not secure", "licensing is viral", "I know this guy that once used open source and got sued" etc. Also you think, you can do better on your own.

  2. Waking up: you are doing this great PNG display library for months now and you have only 20 more known bugs to fix before you can start working on your great SSL implementation then some friend comes by and asks "have you tried libPNG and OpenSSL?". You try them and they seams to work. You start reading licenses.

  3. It's a wonderful world v1.0: The sun shines again. Now you know much about open source licenses. You are using as much open source as you can. In fact it's something like a big well and you can take as much as you like right? You start to understand some of the ideas behind. You also have a strong opinion about how those projects should be managed and what needs to be done next there.

  4. Open your heart: It's a big well but wouldn't it be great if you add something to it? You are looking at your own projects and start thinking - "should I open them?". You decide to try with caution. You open couple of your most unimportant projects. You select them because no one have time for them in hope that "it's a big community, there should be someone willing to add improvements". Now you consider yourself a full grown open source citizen and so is your company.

  5. No love for me: No one wants to work on your open projects. In fact no one cares. You are wondering what's wrong with the community. Looking at other open projects and comparing them to your own, you can see some differences. Maintaining an open source is not an easy task. Getting support from community comes with a price. And if you don't like and care for your projects, others won't too.

  6. It's a wonderful world v2.0: You start opening only your significant projects. Selecting those you are sure many people will find useful. It is important for those projects to have some unique features and/or be clearly better than similar open source projects. They need to be well documented, with clear license and build instructions. Some samples are useful too. Once you have that you will get community support. You understand that open source doesn't necessarily mean nobody gets paid to work on it. It just means it is open for others to look at the code and use it, and they might contribute. Now you can decide should you be the main contributor or just step out and leave it to the world to handle from now on.
     

Friday, September 5, 2008

Announcing Demosten's Perl distribution for Win32

Today I published the first version of my Perl distribution for Windows. My idea was to create a distribution which could fully take advantage of CPAN. This means that you need a C/C++ compiler and make program together with some Unix tools like gzip, tar, less etc. I've added a Perl Scripting Engine (thanks to Atanas Kolev). The distribution also contains several simple scripts to provide a documentation with TOC and to add and remove directories to/from path environment. There is also a simple program written to relocate perl and the compiler to specified directories. The distribution has no additional license limitations and all our code is distributed with the same license as Perl itself.

Links:
Official homepage: http://dnperl.sf.net/
Downloads at project home page: http://sourceforge.net/projects/dnperl/

Sunday, August 17, 2008

Designing the next generation of CDRfe

(last updated 17.August.2008)

I've finally found some free time to think about the next version of CDRfe. This is one of my favorite own programs, but in time I realized that this way of creating C++ closed source software will lead us nowhere. So I'm thinking about designing the next generation of CDRtools frontend. The planning is on really early stage (like some thoughts on a single sheet of paper) and I decide this time to involve the community. I realize this will take much more time than designing it by myself, but I believe the outcome worth it! So here are my general ideas about it:
  • We'll start codding from scratch (will use only knowledge gathered in CDRfe development, but not a single line of code)
  • We'll use some platform independent language like Java which will allow using CDRfe on other OS-es too.
  • Next generation version will combine VisualISO with CDRfe into a single program.
  • I'm planning to include remote recording functionality. This is when you have a single recorder and many clients
  • The new project will be an open source and I'll be very careful with licenses this time when it comes to using external source code and/or libraries
  • We might also select a better name of the project. If you have an idea please post a comment or send me an e-mail.
Now I'll go deeper into the technical part:

I'm planning to use a Rich client platform instead of doing everything on my own. The best candidates are: Eclipse, Netbeans and Mozilla. If you have other ideas, please post a comment or drop me an e-mail at demosten@gmail.com. At the moment my favorite is Eclipse. Advantages of using such a platform are: you have much work done and tested and we could concentrate on the actual functionality instead of for example user interface. So the software could be delivered faster and better tested. Disadvantage I can think of is that it will be much bigger and in general slower.

I need your help! I'm looking for:
  • Fresh ideas what to include inside the next generation version as functionality, usability, code libraries etc.
  • Codding help. I'll need it at moment a concrete Rich client platform and programming language is selected and the work could start. The project will be hosted probably on sourseforge.net and will have a repository and so on.
  • Designer's help: this version should look great :)
If you can help please do it! Post a comment here or drop me an e-mail at demosten@gmail.com. I'll update this post in time so come and check it from time to time.