Technical part:
I worked a lot to decode the data files (scx,rpk..) and I developed some proof-of-concept
(they do not have, how should I put it, proper production code quality
(if I would have done this at work, well let's just say code review would have been quite a shaming))
little programs using the knowledge, this includes: (available from the main SlrrExhaustive download)
-Spl editor and map viewer (testing the concepts of RPK loading, and the spline .spl files define)
-Extra Part aligner (this includes putting together the car as Slrr would through slot attachments)
-UV editor (also, putting the car together properly so that the projected textures would line up)
-The map editor can load maps in POLY or PHYS editor mode which can be used to move
around vertices in a map or its collision
These programs are not made for general public use but I did not want to withhold
anything they may come in handy for someone someday.
The general model for the datafiles is usable the sourcecode gets problematic in the tools.
This was done in (old)c# because well it was easier, also all the sources are provided
(under the "Dont Be a Dick" Public License (https://www.dbad-license.org/)).
On this page I will provide the data file structures (as much as I found out)
which was used to implement the tools
(the c# source for datafile manipulation is understandable and serves as an example
implementation (not all exceptions and common errors are listed here but the c# source is tested)).
c# source for file manipulation
Scx files:
There are two versions that Slrr uses for storing 3D model data:
Scx v3 file structure (fields in order of appearance (in a valid file)):
ScxV3
{
char[4] Signature = "INVO"
int Version = 3
ModelDefV3[] models //as many as can be also there may be an int
at the end of the file that is 0
}
ModelDefV3
{
MaterialV3
int vertexCount
VertexDataV3 vertexData[vertexCount]
int faceCount //triangle count
int IndicesV3[faceCount*3]
}
MaterialV3
{
int size
float DiffuseColorR
float DiffuseColorG
float DiffuseColorB
float Opacity
float SpecularColorR
float SpecularColorG
float SpecularColorB
float SpecularColorWeight
float GlossinessWeight
int Flags
{
int flagAlphaOpacity = 0x00000001;
int flagDiffuseBlend = 0x00000100;
int flagVertexColorBlend = 0x00000002;
int flagLayer2VertexColorBlend = 0x00001000;
int flagLayer2BlendByAlpha = 0x00000040;
}
short diffuseMapIndex //Which texture corresponds to which mapindex is set in the
RenderRef definition part as a list of texture typeIDs in a RPK file referencing the given scx file
short bumpMapIndex
short specularMapIndex
short reflectionMapIndex
short diffuseLayer2MapIndex
short unkownMapIndex2
short illuminationIndex
short unkownMapIndex3
int oneVertexSize //in the next VertexDataV3
int unkownInt1
short unkownShort1
short diffuseMixFirstMapChannel //Channel is the index of UV that should be used
short diffuseMixSecondMapChannel
short bumpMapChannel
short specularMapChanel
short unkownShort2
int unkownInt2
int unkownInt3
int unkownInt4
float illuminationColorR
float illuminationColorG
float illuminationColorB
char materialName[32]
}
VertexDataV3// the size of this part is set in the MaterialV3 and can differ
depending on the fields defined but the order of them is fixed so if an
UV channel 3 should be defined all the field above must be as well
{
float vertexCoordX
float vertexCoordY
float vertexCoordZ
float vertexNormalX
float vertexNormalY
float vertexNormalZ
float uVChannel1X
float uVChannel1Y
float uVChannel2X
float uVChannel2Y
byte vertexColorB
byte vertexColorG
byte vertexColorR
byte vertexColorA
int unkown1
int unkown2
float uVChannel3X
float uVChannel3Y
int unkown3
}
Scx v4 file structure (fields in order of appearance (in a valid file)):
ScxV4
{
char[4] Signature = "INVO"
int Version = 4
int headerEntriesCount
HeaderEntry headerEntries[headerEntriesCount] // the order of these definitions
implicitly declares the logical connection between entries
(a mesh can only have one material definition...)
byte data[] //header entries point into this
}
HeaderEntry
{
int entryType
{
FaceIndices = 5, //FaceDef
MaterialDef = 0, //MaterialV4
HardSurfaceVertexDefinition = 1, //HardSurfaceDef
VertexData = 4, //VertexDataV4
SparseBoneIndexList = 2, //BoneList
BoneIndexList = 3 //BoneList
}
int entryOffsetInFile
}
MaterialV4
{
int unkownZero = 0
int size //of block
int flags
int unkownZero2 = 0
int EntriesCount
MaterialEntry entries[EntriesCount]
}
MaterialEntry
{
ushort unknownFlag
ushort Type
byte data[DataSize]
//DataSize is dependant on the Type:
//Type == 2048 => DataSize = 32 (char[32])
//Type == 0 => DataSize = 4 (byte RGBA)
//Type == 256 => DataSize = 4 (float intensity)
//Type == 1536 => DataSize = 28 (MapDefinitionEntry)
//Type == 1280 => DataSize = 4 (int)
//You can also deduce semantic meaning
(for this I used the TypeSwaped = (ushort)((Type >> 8) | (Type << 8)))
//if (unknownFlag == 0 && TypeSwapped == 0)
// return "Diffuse Color";
//if (unknownFlag == 2 && TypeSwapped == 0)
// return "Illumination Color";
//if (unknownFlag == 1 && TypeSwapped == 0)
// return "Specular Color";
//if (unknownFlag == 0 && TypeSwapped == 1)
// return "Glossiness";
//if (unknownFlag == 1 && TypeSwapped == 1)
// return "Reflection Percent";
//if (unknownFlag == 2 && TypeSwapped == 1)
// return "Bump Percent";
//if (unknownFlag == 0 && TypeSwapped == 6)
// return "Diffuse Map";
//if (unknownFlag == 1 && TypeSwapped == 6)
// return "Diffuse Mix Second";
//if (unknownFlag == 2 && TypeSwapped == 6)
// return "Bump Map";
//if (unknownFlag == 3 && TypeSwapped == 6)
// return "Reflection Map";
//if (unknownFlag == 4 && TypeSwapped == 6)
// return "Illumination Map";
//if (unknownFlag == 0 && TypeSwapped == 8)
// return "Material Name";
}
MapDefinitionEntry
{
int textureIndex //this index is taken from the texture TypeID list defined
in a RPK that references the scx file containing this map definition record
int mapChannel //UV used
int TillingFlags//3=UTileVTile 0=UMirrorVMirror
float TillingU
float TillingV
float OffsetU
float OffsetV
//if the next entry is a float intensity MaterailEntry it is the
weight of this Map (like reflection strength)
}
FaceDef
{
int type
int size //of block
int numOfShorts
ushort indices[numOfShorts]//I must point out that these are ushorts meaning if there are
more than 0xffff (65535) vertices in a given vertexDataV4 indices can overflow which in
turn can lead to crashes because of the extreme triangles
//well actually slrr even used these as shorts when projecting decals onto parts
(on collision or by painting) so there could only be 0xffff/2 vertices in a mesh
(a model is made out of meshes)
//technically:
//004F2DA1 (offset in Slrr.exe): two loops looping through all indices and hence
all vertices this can calculate bad pointers...
//the loop checks a triangle (3 offsets are calculated but MOVSX is used to
load EDI which is "singextend" meaning it is cast to a short from a ushort....
this is very bad EDI can move backwards(even before the base pointer) in the buffer
if the index is greater than MAX_USHORT/2 which can happen ... that's not that many vertices,
all 6 (3 in each loop) MOVSX instructions should be MOVZX (zero extend))
}
VertexDataV4
{
int type
int size //of block
int numberOfVertices//will be indexed with ushorts in FaceDef so it should not be > 0xffff
int vertexType//flags declaring which fields are defined in the VertexData
//PossibleFeatures: (in order of possible appearance)
//Position = 0x1, float[3] XYZ floats
//BoneWeightNumIs0 = 0x2, 0 weight-float written (one is implicit 1-Sum(WrittenWeights) = 1)
//BoneWeightNumIs1 = 0x4, 1 weight-float written (one is implicit 1-Sum(WrittenWeights))
//BoneWeightNumIs2 = 0x8, 2 weight-floats written (one is implicit 1-Sum(WrittenWeights))
//BoneWeightNumIs3 = 0x10, 3 weight-floats written (one is implicit 1-Sum(WrittenWeights))
//BoneIndRef = 0x20, byte[4] (four indices to the BoneIndexList)
//Normal = 0x40, float[3] XYZ floats
//VertexIllumination = 0x80, byte[4] RGBA
//VertexColor = 0x100, byte[4] RGBA
//UV1 = 0x200, float[2] UV
//UV2 = 0x400, float[2] UV
//UV3 = 0x800, float[2] UV
//BumpMapNormal = 0x40000, float[3] XYZ floats
byte vertexData[(size - 16)] //vertex data defined in accordance with the vertexType
}
HardSurfaceDef //if this entry is present the current mesh is a hard surface mesh meaning
there won"t be any bone references in the vertexDatas
{
int type
int size
int unkownInt = 0
}
BoneList
{
int type
int size
int unkownInt = 0
int BoneIndexList[((size - (3 * 4)) / 4)]
}
Rpk files:
These files are used to define the relations between other data files
(telling a Part script which cfg to use and which scx to use with which textures...)
Rpk
{
Header header
ExternalReference extRefs[header.externalReferencesCount]
EntriesHeader entriesHeader[]
ResEntry resEntries[entriesHeader.entriesCount]
byte rsdData[]
}
Header
{
char signature[4] = "RPAK"
int version512 = 512
int externalReferencesCount
int externalReferencesUnkownZero = 0
}
ExternalReference
{
short unkownIndexZero = 0
short indexOfReference
char referenceString[60] //padded with 0-s (at the end)
}
EntriesHeader
{
int entriesSize;//resEntries size in bytes
int entriesCount
int entriesType1Count // number of entries with typeOfEntry == 1
int entriesNonType1Count
}
ResEntry
{
int superID
int typeID// it is called Type ID because it only needs to be unique in
combination with the value of typeOfEntry
byte typeOfEntry
byte additionalType
float isParentCompatible
int fileOffsetOfRSD
int RSDLength
byte aliasLength
char alias[aliasLength] //despite the length defined this must be zeroterminated
(size includes the zero)
(//optional
float floatBounds[12]// it is present if additionalType & 1 != 0
this is used in the spatial data structure used to define maps
)
}
RSDEntry
{
InnerRsdEntry innerEntries[]//as many as the corresponding RSDLength can hold
}
InnerRsdEntry can be one of; ICFGInnerEntry, XCFGInnerEntry, RSDInnerEntry,
StringInnerEntry, EXTPInnerEntry, NamedDataArray, InnerPolyEntry, InnerPhysEntry
ICFGInnerEntry //Internal cfg entry: a cfg file defined directly in the rpk
{
char signature[4] = "ICFG"
int lengthOfData
char cfgData[lengthOfData]// \0 delimited list of strings (2 strings observed)
defining a common cfg with a key
}
XCFGInnerEntry //reference cfg
{
char signature[4] = "XCFG"
int lengthOfData
char cfgData[lengthOfData]// \0 delimited list of strings the first one is usually
empty the second one is a key, a whitespace and than a file path relative to the
slrr root directory to a cfg file
}
RSDInnerEntry
{
char signature[4] = "RSD\0"
int lengthOfData
char stringData[lengthOfData]// string data used mainly to define map "entry points"
//expect NamedDatas and the spatial structure definition of a map (as in track not as in picture)
}
StringInnerEntry
{
char stringData[] //as much as the current RSD entry can hold
(ResEntry.RSDLength (or as much as is left of it))
}
EXTPInnerEntry
{
char signature[4] = "EXTP"
int lengthOfData = 4
int data//only 3 observed it is also the number of the root NamedDataArrays defined
but that can be coincidence
}
NamedDataArray
{
char signature[4] = "\0\0\0\0"
int numOfNamedDatas
NamedData namedDatas[numOfNamedDatas]//these represent the spatial tree data structure
(not a BSP, some axis aligned thing not exactly an octree but definitely some bounding volume hierarchy)
}
NamedData
{
int TypeID
int SuperID
byte typeOfEntry
byte additionalType
float isParentCompatible
int offset//in file, references NamedDataArray, InnerPolyEntry or InnerPhysEntry entries
not always just one but uniquely; only one NamedData references a given array or poly or phys
int sizeAtOffset
byte nameLength
char name[nameLength]//just as with the RES entries
float boundingBoxX//in range -50000 .. 50000 positioning the centre of the box
float boundingBoxY//in range -50000 .. 50000
float boundingBoxZ//in range -50000 .. 50000
float boundingBoxHalfWidthX//in range 0 .. 50000 generally small values
float boundingBoxHalfWidthY//in range 0 .. 50000 generally small values
float boundingBoxHalfWidthZ//in range 0 .. 50000 generally small values
float unkownData7// only 0 seen
float unkownData8// only 0 seen
float unkownData9// only 0 seen
float unkownData10// only 0 seen
float unkownData11// only 0 seen
float unkownData12// only 0 seen
}
InnerPolyEntry
{
char signature[4] = "POLY"
int size
int unkownCount1
int meshCount
Mesh meshes[meshCount]
}
Mesh
{
int textureIndex
int verticesCount
int triangleCount
VertexData vertexData[verticesCount]
int Indices[triangleCount*3]
}
VertexData
{
float coordX
float coordY
float coordZ
float normalX
float normalY
float normalZ
byte colorR
byte colorG
byte colorB
byte colorA
byte illuminationR
byte illuminationG
byte illuminationB
byte illuminationA
float uVChannel1U
float uVChannel1V
float uVChannel2U
float uVChannel2V
float uVChannel3U
float uVChannel3V
}
InnerPhysEntry
{
char signature[4] = "PHYS"
int size
int indexChunkCount
IndexChunk indexChunks[indexChunkCount]
int vertexChunkCount
VertexChunk vertexChunks[vertexChunkCount]
}
IndexChunk
{
int unkownInt1//probably the material...
int triIndices[3]
float normalX
float normalY
float normalZ
}
VertexChunk
{
float vertexX
float vertexY
float vertexZ
}
Class files:
The compiled script files of slrr this uses a custom java implementation
it was developed with Flex(,Bison) ... if someone wants a challenge (to investigate the saved AST)
I only figured out the bare minimum that is needed to change the rpk references,
class and package names in the .class files without .java sources
Class
{
TUFA definedClasses[]//sequentially
}
TUFA
{
char signature[4] = "TUFA"
int unkownData1
int unkownData2
SignaturedSizedScriptEntry entries[]//as many as can be till the next TUFA entry,
can be SignaturedSizedScriptEntry or CONSEntry
}
SignaturedSizedScriptEntry
{
char signature[4]
int size;
byte data[size]
}
CONSEntry
{
char signature[4] = "CONS"
int size;
int entriesCount
ConstantEntry entries[entriesCount] //can be ConstantEntry, NameConstantEntry or RpkRefConstant,
the full class name(including the package) is the first NameConstantEntry
the full parent class name is the second
}
ConstantEntry
{
int ID
byte[] data// size is determined by the ID:
//ID == 0 => size = (the first int in data)+4
//ID == 4 => size = 4
//ID == 7 or 3 or 5 => size = 8
//data is generally an int[] referencing other ConstantEntries by their indices in the
given constant table if flatted out it will give a function reference or other string
}
NameConstantEntry
{
int ID = 0
int stringLength
char name[stringLength]
}
RpkRefConstant
{
int ID = 3
int RPKnameIndexInConstantTable//the index of a NameConstantEntry in the current
CONS that will be the rpk referenced (the file path)
int TypeIdInRPK
}
Other assorted info about Slrr:
In the cfgs for vehicles and parts stockpart 0x00000001 line means that when creating
a dummy model try and attach the part with typeID 0x00000001 (this is used when
creating the cars roaming the city) creating a dummy car:
dummycar = new GameRef();
dummycar.create_native( map, new GameRef(dummyID), "0,-10000,0,0,0,0", "dummycar" );
here stockpart references will be attached
In the cfgs for vehicles and parts the texture 0x00000001 line means that the texture 0x00000001
(which should be in the textureID list of the corresponding renderRef definition) will be replaced
when painting is applied (or the setTexture method)
this can be used to create parts with paintable reflection or illumination textures which is quite cool.
Slrr manages RenderRefs competently but scripts are difficult for it this is the reason
why in the demonstration "modded almost to hell" version of Slrr I created all the RenderRef lookup tables
(to delay the loading of the corresponding classes);
once a script is loaded (meaning an instance of one of the defined classes is created) it can not be unloaded
and this is why slrr will always crash eventually; it is a 32bit application so it can only address
~4Gb memory which will easily run out because every part has a unique class,
of course as demonstrated in the videos this can be pushed out significantly
(using the renderRef lookups), and this is why all the ExtraParts use the same script.
The silent crashes (ones that do not produce a line in the error.log) are
(if you can rule out the obvious missing resource thing and class redefinition
(which can be easily checked using the file type model))
almost always caused by
the GC freeing up a script instance from under another script-thread,
multi threading is very dangerous in Slrr
(Miran's added little camera icon to the navigator
(which is updated from a thread created by the Track class) was a frequent cause of crashes).
The game uses standard fixed function pipeline directx9
The .spl file spline row format (%LF is float %d is int) is:
(%lf %lf %lf) (%lf %lf %lf) %lf %lf %d %lf %lf %d:
(pos XYZ), (normal XYZ), splineWidth, targetVelocity, I do not know the rest
the two ints are used as bools
the list of first two vectors define a closed cubic Hermite spline.
Slrr can not load more than 256 RPKs and there are at least 5 reserved slots that get filled by the exe
itself and not by loading a library this modded version loads 248 (I think), and that is pretty much the maximum
possible.
Imma send you my research text file as anyway i'm not really going to do anything else with it anymore, and an old exe tool i got, i'm sure it will help you defining some of the unknown values.
For example, i think you didnt add(but i might've just not seen it) self-illumination percentage, self-illumination color and self-illumination texture, the backface culling option amd something else right now i just dont remember.
Anyway you'll see it by your own and eventually add stuff to your research
2. Posted by HardFFlip 2018-02-22 16:18
Any chance to make the scx v4 export plugin for newer versions of 3ds Max? 2015 or higher, for example. Because sometimes exporting from 3ds Max 2005 like pain in da butt.
3. Posted by amilmand 2018-02-22 19:37 Illumination is there but if you can extend it surely more knowledge is always better.
Well the definition is there and can be used to make an exporter for someone familiar with the plugin architecture of 3dsmax (I would probably expect a blender plugin instead but who knows) I myself really doubt to be able to find the time to learn this (since not every aspect of the scx file format can be edited in 3dsmax, thinking about the flags of a material in particular, the plugin needs to define some custom modifier and/or material to properly set every property of the resulting scx)
Nevertheless I maintain that, although archaic, using 3dsmax5 and the original exporters are preventing a lot of issues exactly because of the limitations it holds the modder to, the avarege quality of mods is low enough with "only" needing to care to make the cfg, rpk, and java definitions without errors, if one would more easily be able to make bad meshes aswell whether because of a badly written exporter or simply because the limitations were stripped... I dont know, it seems to be a far too complicated system to integrate into already
Total : 3, on page: 3