Portability considerations for Linux and Mac

For questions or suggestions related to the source code.

Portability considerations for Linux and Mac

Postby michelle on Thu, 17th Jul 2008, 16:22

I have prepared the source code from the beginning to work platform-independently. However, external libraries are not portable, and one of the things that need to be done is to collect the necessary platform-specific files. I will guide you on how to do that using SharpDevelop or MonoDevelop.

But even before this can be done, there are still problems in porting the program to other OSs. One thing is file names and file systems.

There are several occasions where path names are constructed in the program. The example code System.IO.Path.Combine(TrainFolder, TrainName) is used to combine the train folder (e.g. C:\Program Files\openBVE\Train) with the train name (e.g. 113_6), resulting in C:\Program Files\openBVE\Train\113-6 on Windows. The same function will instead use the slash character on Linux and Mac, so it is a platform-safe function, and I solely use these kinds of functions provided by the .NET framework.

However, in object files, a LoadTexture command might reference the relative file name ..\texture.bmp, which when combined with the base object folder (e.g. C:\Program Files\openBVE\Railway\Object) will yield C:\Program Files\openBVE\Railway\Object\..\texture.bmp. When accessing this file, the file will be correctly found under C:\Program Files\openBVE\Railway\texture.bmp, thus resolving the ".." part. On Linux or Mac, this could instead become something like /openBVE/Railway/..\texture.bmp, which might not be processable in those OS's due to the backslash character.

I have therefore started to correct all path parts read from text files in order to convert from the backslash character to the slash character on Linux and Mac. This means that the game EXE itself cannot be cross-platform, as conditional compilation is required (which is required anyway, just to mention that).

What I want to do with this thread is to collect system-specific problems that I have missed to see, and which might prevent using the program on other OSs.

Things I have investigated and prepared so far:

Mac OS X:
The path separation character is the slash (/). Path parts read from text files will replace their backslash characters to slash characters.

Linux:
The path separation character is the slash (/). Path parts read from text files will replace their backslash characters to slash characters.
Furthermore, file names are case-sensitive, which might prevent the program to find files referenced in a route file when the actual files use a different case for some characters. The solution already partially implemented is to test if the file exist in the spelling used, and if not, the path will be split into two parts: The directory and the pure file name. First, the directory name will be recursively edited with the process described hereby. Second, once the directory name is valid, all files in the directory are checked if they match case-insensitively. Once a file has been found, that file name is returned instead of the original one, so that the file can be found by subsequeny processes. The directory is recursively edited in the same way by checking the parent directories until a match is found. If no match is found in any of the processes, then the file does not exist anyway regardless of the spelling

If there are any other considerations or any parts I mentioned here unnecessary, please tell me.

And of course, almost needless to say, BVE 4 ATS DLL plugins cannot work on other OSs but Windows. Trains will still be usable, just without the plugin.
User avatar
michelle
Site Admin
 
Posts: 1139
Joined: Mon, 14th Apr 2008, 20:36

Re: Portability considerations for Linux and Mac

Postby afarlie on Thu, 17th Jul 2008, 20:03

And of course, almost needless to say, BVE 4 ATS DLL plugins cannot work on other OSs but Windows. Trains will still be usable, just without the plugin.


Would it be possible, at least in terms of OpenBVE to provide an abillity to use a .NET assembly type plugin,
with a simmilar interface to that currently used?
afarlie
 
Posts: 43
Joined: Mon, 14th Apr 2008, 22:57

Re: Portability considerations for Linux and Mac

Postby michelle on Thu, 17th Jul 2008, 22:01

I would not want to do that and have already announced so. Using .NET assemblies would of course be a portable solution. However, I think openBVE should be open to edit by anyone, not just programmers. The community of BVE is still somewhat of a niche, and you can count the developers having written plugins for BVE 4 on at most two hands, I think, and that even world-wide. For openBVE, the situation will probably not be better. I think the better approach would be using scrips which can be edited in any text editor, making development also possible for non-programmers. The .NET framework provides a nice namespace called CodeDom, which is able to compile its internal structures directly into executable files or libraries, be it to the hard disk or in-memory. Using this approach, there would be no performance impact compared to interpreting a fairly complex script on a per-frame basis, although I could make even that quite fast. From my point of view, using a scripting language (of course a custom one) combined with on-the-fly compilation is the best approach to be used by anyone.

Let's focus on portability issues again.
User avatar
michelle
Site Admin
 
Posts: 1139
Joined: Mon, 14th Apr 2008, 20:36

Postby afarlie on Sat, 19th Jul 2008, 15:25

Portability issues -

- Open GL libraries (type and location)
- File paths (noted above)
- Location of display 'device' in relation to machine actually running code
afarlie
 
Posts: 43
Joined: Mon, 14th Apr 2008, 22:57

Postby amac on Mon, 21st Jul 2008, 20:07

Just a note to say that I've compiled the openbve code on Mono/Linux, and had a tiny measure of success running it.

Screenshot

The most obvious problems have been case issues; "Limits" and "Signals" in particular. Creating symlinks worked round that in the meantime, though I did come across a Train.dat / train.dat issue too ("solved" in the same way)

The only track I've got any visuals or audible results with is the BVE2 Uchibo track; if I select "ignore" to the mountain of "texture not found" type messages, driving the train seems to work pretty much as expected I think, other than the obvious lack of most textures.

Sounds seem to be mostly working, too. On any other track I've tried, I just seem to end up with a black OpenBVE window, and not much happening.

I'm almost certainly not the best person to work on this; I'm not a programmer (a sysadmin), this is the first Mono program I've ever compiled and I'm a complete BVE novice (hadn't used it until it at all until it became possible to run OpenBVE with WINE)!

This post is mostly just an encouragement for someone a bit more knowledgeable on the mono side to work on getting this running 100% on Linux, though I'm willing to try suggestions as my limited time permits...
amac
 
Posts: 8
Joined: Mon, 21st Jul 2008, 19:35

Postby michelle on Mon, 21st Jul 2008, 20:17

Well, this seems like it's basically working on Linux then. The problems with no textures are then, as the error messages you get suggest, problems with finding the files. I will check the code again whose purpose is to find files case-insensitively on Linux, but it is a bit hard to test under Windows. Additionally, the screendoor transparency does not seem to work. Internally, it is achieved by loading the images and editing them in-memory. It could be that the internal format returned by the image loading function is different in Linux, resulting in wrong editing. This will be even harder to debug for me as I do not have Linux (and do not intent to), but I will try finding the causes of these problems. Or, by taking a look at the ATS, ATS POWER etc. messages, it rather seems that alpha transparency is not supported on your system, which is rather odd...

Try using the nearest neighbor filter by adjusting the settings.cfg and turn off anisotropic filtering if enabled. Could it be that alpha is dithered on your system?

Still, if the program did not even crash, it seems like a very promising start after all...
User avatar
michelle
Site Admin
 
Posts: 1139
Joined: Mon, 14th Apr 2008, 20:36

Postby michelle on Mon, 21st Jul 2008, 20:28

However, still some questions:

If you get error messages about textures not found, could you check against which spelling is returned in the message box and compare that to the actual file name path? What is the difference and thus reason why the files are not found?

Could you give me an example of a texture file not found and its complete path as you would phrase it under Linux?

Not that it is fair to compare yet, but what kind of frame rates do you get compared with using WINE on the same route?
User avatar
michelle
Site Admin
 
Posts: 1139
Joined: Mon, 14th Apr 2008, 20:36

Postby amac on Mon, 21st Jul 2008, 21:03

michelle wrote:However, still some questions:
If you get error messages about textures not found, could you check against which spelling is returned in the message box and compare that to the actual file name path? What is the difference and thus reason why the files are not found?
Could you give me an example of a texture file not found and its complete path as you would phrase it under Linux?


(excuse the ridiculous path!)

Line 11 in /tmp/junk/real/openbve/0.7.0/Multi-platform\ via\ Mono/bin/Release/Compatibility/signals/signal_3_0.csv

That path is fine (though only because I've symlinked "Signals" to "signals"), but line 11 says

LoadTexture,graphics\signal_3.png

The problem there is the backslash; if I change it to a forward slash that particular error disappears.

Not that it is fair to compare yet, but what kind of frame rates do you get compared with using WINE on the same route?


Around 20fps in wine, ~80 fps native. I would expect that difference to drop a bit once all those other textures get loaded of course!

Setting the nearestneighbor option didn't change anything that I could see. I do a quite lot of work on the FlightGear project and so am confident that my OpenGL setup is fully functional for "normal" Linux programs - it also works fine for OpenGL software running under WINE.

Anyway - like you, I'm quite impressed that it runs and really with remarkably little effort at my end; most of the effort was in getting the Tao framework installed (I already had native OpenAL/SDL/OpenGL etc all in place) and in figuring out how to compile a mono project. It'll be great to have such a nice train sim available fully open source and running native on Linux!
amac
 
Posts: 8
Joined: Mon, 21st Jul 2008, 19:35

Postby michelle on Mon, 21st Jul 2008, 22:28

Ok, then I will fix that remaining backslash problem and will also try to debug as if my computer was Linux... Then I will post a corrected source code within a day or two.

EDIT: Did you use the LINUX conditional compilation flag?
User avatar
michelle
Site Admin
 
Posts: 1139
Joined: Mon, 14th Apr 2008, 20:36

Postby rlazur on Tue, 22nd Jul 2008, 02:17

Okay, path issues taken care of with libraries.

Next up:

Code: Select all
Building Project: OpenBve (Debug)
Performing main compilation...
Compilation failed: 4 error(s), 4 warnings

openbve0700source/Multi-platform via Mono/BveCsvRwRouteParser.cs(265,20): warning CS0219: The variable `SignalGraphicsFolder' is assigned but its value is never used
openbve0700source/Multi-platform via Mono/BveCsvRwRouteParser.cs(936,72): error CS0117: `OpenBve.Interface' does not contain a definition for `GetCorrectedPathSeparation'
openbve0700source/Multi-platform via Mono/Interface.cs(5,27): (Location of the symbol related to previous error)
openbve0700source/Multi-platform via Mono/BveCsvRwRouteParser.cs(1387,101): error CS0117: `OpenBve.Interface' does not contain a definition for `GetCorrectedPathSeparation'
openbve0700source/Multi-platform via Mono/Interface.cs(5,27): (Location of the symbol related to previous error)
openbve0700source/Multi-platform via Mono/BveCsvRwRouteParser.cs(1395,98): error CS0117: `OpenBve.Interface' does not contain a definition for `GetCorrectedPathSeparation'
openbve0700source/Multi-platform via Mono/Interface.cs(5,27): (Location of the symbol related to previous error)
openbve0700source/Multi-platform via Mono/BveCsvRwRouteParser.cs(2701,39): error CS0117: `OpenBve.Interface' does not contain a definition for `GetCorrectedFolderName'
openbve0700source/Multi-platform via Mono/Interface.cs(5,27): (Location of the symbol related to previous error)
openbve0700source/Multi-platform via Mono/BveCsvRwRouteParser.cs(3104,28): warning CS0219: The variable `dy' is assigned but its value is never used
openbve0700source/Multi-platform via Mono/BveCsvRwRouteParser.cs(2746,41): warning CS0168: The variable `Temp' is declared but never used
openbve0700source/Multi-platform via Mono/BveCsvRwRouteParser.cs(2744,20): warning CS0219: The variable `SignalGraphicsPath' is assigned but its value is never used


Build complete -- 4 errors, 4 warnings

---------------------- Done ----------------------
Build: 4 errors, 4 warnings


Hmm, it's in interface.cs I see. I couldn't get the pre-compiler statements to work on my macbook. I removed them and got a successful build. That is not proper. I tried just having 'MAC' or 'MACX' in its own #elif but it didn't help. I wonder if an #else would work?

Also to consider -- I'm not sure if libalut is standard on Macs (openal is, but not libalut.so.0?). Does anyone know the answer to this? So I can't run the binary just yet because of this.

There are unknown symbols trying to build freealut so I'm not sure. Maybe we need to check if sound should be enabled or not if this is true?
rlazur
 
Posts: 23
Joined: Tue, 27th May 2008, 21:53

Postby amac on Tue, 22nd Jul 2008, 09:03

michelle wrote:EDIT: Did you use the LINUX conditional compilation flag?


As far as I know, yes. Under Project Options, and under the relevant DEBUG/RELEASE configurations I added LINUX to the "Define Symbols" list. As I mentioned I know nothing about handling this sort of thing under mono, but since the compile failed without that I assumed it to be correct.

Regarding OSX and alut... Flightgear requires alut and runs on OSX - this post from one of our Mac developers might give some pointers.
amac
 
Posts: 8
Joined: Mon, 21st Jul 2008, 19:35

Postby rlazur on Tue, 22nd Jul 2008, 12:06

amac wrote:Regarding OSX and alut... Flightgear requires alut and runs on OSX - this post from one of our Mac developers might give some pointers.


Heh, that's one of the few discussions I was able to find on the subject last night. :)
rlazur
 
Posts: 23
Joined: Tue, 27th May 2008, 21:53

Postby michelle on Tue, 22nd Jul 2008, 12:51

@rlazur: If you want to compile for Mac, set the MAC conditional compilation flag, otherwise you will get errors about missing function declarations. I don't know how to do it via MonoDevelop, but it can't be all that different compared to other development environments. Try to open the properties of the OpenBve project (not the solution) and browse your way through a tab usually called Compile. Somewhere, there is text field where you can enter conditional combination symbols. On SharpDevelop, there was already one called TRACE defined. Just change that to "TRACE,MAC". Or if no TRACE is present, simply "MAC". Do not remove any parts of the source code as it will become very likely that it won't work afterward.

@amac: What you did seems to be about correct.

@both: openBVE should theoretically take care of the different path conventions, so no additional "libraries", whatever they are, should be required. If you still spot additional problems, I will do my best to fix them.
User avatar
michelle
Site Admin
 
Posts: 1139
Joined: Mon, 14th Apr 2008, 20:36

Postby rlazur on Wed, 23rd Jul 2008, 04:22

Thanks, it builds.

I'm still trying to figure out the libalut.so.0 issue. Moving alut.h around doesn't seem to effect building freealut, and I cannot seem to find a dowloadable binary for the life of me.
rlazur
 
Posts: 23
Joined: Tue, 27th May 2008, 21:53

Postby michelle on Wed, 23rd Jul 2008, 16:36

I have uploaded a source code bugfix to the homepage. The functions responsible for replacing the backslash into the platform-specific character (Linux and Mac) and the functions for finding files case-insensitively (Linux only) all had some problems and basically did not work at all. I have a hard time simulating the conditions under Windows, but I could at least fix these problems. You can extract just the Interface.cs file from the source code download and put it into your source code directory without changing any of your project settings, and then simply recompile. It should (hopefully) fix the file-finding problems and should thus allow textures to be loaded correctly. But report back if it does still not work...
User avatar
michelle
Site Admin
 
Posts: 1139
Joined: Mon, 14th Apr 2008, 20:36

Next

Return to Source code

Who is online

Users browsing this forum: No registered users and 1 guest