Recently I’ve figured out how to let people paint textures of objects in a 3D scene, as well as create their own sound effects in a simplified manner. That’s great, but people would still be stuck with the models that I design. To introduce more creative opportunities, I pondered what it would take for people to make their own 3D models.

I only wanted to work with the mesh object itself. The obj file format includes additional information like texture coordinates, lines, faces, etc. Looking back at the Sculpted Prim, the format is specific to only storing control vertices (CVs). My understanding is that connections between CVs are based on a sphere since the way textures are mapped to sculpted prims are spherical. There is an added benefit with the format in regards to length of distance where it can reduce the number of points used from the image to improve performance when rendering objects that are far away. At the time, it was the most ingenious way to introduce a new primitive type to the Second Life platform using the tools and architecture already available.

Benefits of Sculpted Prims

• Images are a native format to the internet
• Formats supporting transparency can visually embed details such as logos
• Various formats support attaching metadata such as EXIF headers
• A MESH can be simple or complex by adjusting the number of control vertices used from the 2D image

With my newfound skills, I may be able to read, modify, and save sculpted prims all within the web browser. I wouldn’t need to use the Phoenix Firestorm Viewer to export my sculpted prims to the collada file format. I wouldn’t need Blender to import the file and export it to the graphics library transition format. I could use all of the existing tools such as Rokuro, Tokoroten, PloppSL, SnurbO’Matic – that is, if I can find them and they still work. On top of that, I have a lot of sculpted prims that I made in the past that I can work with.

Acquire Sculpted Prims

I started going through my inventory in Second Life and downloaded any texture I could find that were specifically for sculpted prims, and had me listed as the creator. With this set of images, I could verify my success wasn’t just a chance with one image, but would be compatible with any sculpted prim regardless of image size. In addition, I would have the Second Life viewer to fall back on to review the actual model mesh versus my interpretation.

I’m sure there are more, but here is a nice variety of sculpted prim images.

Plot Control Vertices

Simplified steps

1. Load the image
2. Read each pixels Red, Green, and Blue values
3. Convert to X, Y, Z coordinates
4. Render a small sphere at the X, Y, Z coordinate
5. Hope that a 3D model made of dots, similar to the original model appears
6. Adjust camera or coordinate mapping for scale, rotation, etc.

An image is two dimensional with x and y coordinates. An image has up to four channels – red, green, blue (RGB), and alpha (transparency/opacity). Each pixel (picture element) is made up of four values representing the intensity of each channel. RGB is mapped to XYZ coordinates in a 3D space. The alpha channel is not used with Sculpted Prims.

The first step was easy. Create a project, add the images, and create a web page to select and display one of the images.

The next step shouldn’t be that hard. Rather an use an html image element, load the image in memory and draw it on a canvas. From there, I can read each pixel. Simple. Except it wasn’t. I was viewing the page from my local file system and ran into an error about the canvas being tainted by cross-origin data.

The next step was to setup a local web server and use localhost. In initialized the project as an npm package, installed the latest version of vite as a development dependency, and crated a start script to run vite and open the page in my web browser. I immediately saw as list of 16,384 control vertices in my console log for the 45 Degree Adapter Arm with values ranging from 0 to 255.

My assumption is that these should probably be floating numbers ranging from negative one to positive one. Let’s ignore that for now. Next is to setup the 3D environment and plot the coordinates in a 3D space.

After a bit of trial and error, I got something to display. I mapped the RGB values to subtract 128, and then divide by 128 to get values between -1 and 1. I also setup each one to display as its original color in the image. This resulted in a bunch of dots that formed a radial gradient between red, green and blue.

I was running into a few problems. It was very slow. It was also overlying the previous objects instead of removing them. A nice warning appeared in the console regarding speed, recommending that I add the “willReadFrequently” attribute to the canvas.

I addressed the warning and was able to get it to go away. However, it had no discernable effect on the speed of rendering. I had to be overlooking something.

I started tackling how the objects were being removed from the scene. I was simply looping through all children and removing them. I decided to create an array of each object and loop through it instead. In addition, I also called the dispose function on the mesh’s geometry and material. The previous points went away when changing images. My comment box appears as a square object, and I was seeing it from the top, appear as dots forming a diamond.

Without the extra objects lingering around, I now had to attack the speed issues. I took a look at the sphere geometry used to represent each point. I had previously looked at a sample for creating spheres that used 32 segments for both the width and the height. Sure, they probably looked like nice spheres close up, but I was focused more on just getting a pixel to display in a 3D space. Just for giggles, I reduced the segments down to 1. Bingo!

Just for added measure, I changed the geometry to use a box to further reduce the mesh. I noticed that corners and certain points had a cluster of points. This made a lot of sense as it was difficult to create hard angles with Sculptured Prims. To work around it, you had to create multiple points at the same location.

Link and Rotate CVs

Looking at a static top-level view of my object felt like great progress. However, I couldn’t discern what some of the other object were. In addition, I wanted to see it in a 3D space – not just a top down view. I had to find a way to visually see it from different angles to see the depth and different sides.

Linking the objects together into one parent object was rather easy with the three.js library. You simply create a new THREE.Object3D() object as the parent, and then add the mesh of each control verticy. I changed my cleanup to loop through the parents children and disposing each ones geometry and material before removing the parent object from the scene. From there, I incremented the rotation of the parent object with each frame render to see all of the control vertices spin together as one object. I now had a good grasp of the actual three dimensional shape as I viewed each object spinning.

Camera Controls

The aspect ratio of the original model is not stored in the image itself. A tall and skinny mesh appears squished when first applied to a cube in Second Life. Once applied to a cube within Second Life, it was up to the creator to scale the cube itself to restore the original aspect ratio. When displaying the models in a 3D space, I’m seeing the same issue. I need to start adding some camera controls to rotate, pan, and zoom in. I also need to add the ability to scale the X, Y, and Z axis of the parent object.

The first thing to do was to render the scene onto a canvas element instead of the whole window. I started getting weird errors about the WebGL context being lost and restored. Eventually it took me down a path where WebGL itself couldn’t be created.

I refreshed, reset the browser, and was considering restarting my operating system. Before I did that, I pushed my commit history to github and then reset hard to the commit before I rendered to the canvas. Odd. Everything was working. This confirmed that the bug was in my code, and not due to the web browser, operating system, or graphics hardware. What could it be?

I tracked it down to what I appended the renderers DOM element onto. Once I changed it back to the document.body, the 3D scene rendered fine.

For anyone who’s familiar with three.js, you’ve probably spotted the problem. “image-3d” is a canvas element. You can’t nest a canvas element within another canvas element. There are two ways to fix this. The first is to change the html so that my image-3d is a div container. The other way is to instantiate the renderer by passing the canvas3d, and remove the appendChild.

With that out of the way, lets create some rudimentary camera controls…

Holy Smoke! I asked ChatGPT to write some code to rotate, pan, and zoom in on the camera. I didn’t expect much. Maybe some algorithms with trigonometry calls to rotate around an object. With only a few lines of code, I thought I didn’t really get anything useful at first. I noticed that it instantiated OrbitControls, but it didn’t do anything with it. I looked into what the class does and found that it wires up mouse events to the rendered DOM object and does all of the calculations to move the camera on your behalf.

I wired up the controls and found the camera was controlled by the mouse just like it said. Everything was intuitive as the controls work like most 3D modeling programs. Just for context:

• Mouse wheel to zoom in/out
• Mouse drag to rotate around the object
• Mouse drag + (shift, control, or command) to pan the camera

Scaling Controls

With camera controls out of the way, its now time to scale the X, Y, and Z coordinates. A very simple process, I used the input range sliders and updated the coordinated vertices parent objects scale. All child objects were updated accordingly. One of the first things I tried it on was the model of a fish.

Connect The Dots

The vertices are great for visualizing what the object should look like, but its still difficult to get a clear idea of what the solid mesh would look like. Some of the models such as the business cards and pie display table just look like dots. It’s time to create a control mesh (aka control net) from the control vertices.

It took a bit of work, but I was able to use BufferGeometry to create a mesh from the control vertices. The 45 adapter arm appeared as if it were a ghost. The business cards displayed only a few faces (triangles).

From my understanding, the sculpt maps are a spherical mesh. The height and width of an image should represent the number of horizontal and vertical segments in a sphere. So each pixel should be able to map to its 2D neighbor in the 3D mesh. The edges are a bit different. The left and right edges need to map to the other side of the image connect them.

If I look at the pixel at x, y in the middle of the image to get its three dimensional coordinates from the RGB channels, then the resulting vertex should connect to the other vertexes represented by neighboring pixels relative to the current pixel:

Relative	Left	Center	Right
Top	x – 1, y – 1	x, y – 1	x + 1, y – 1
Middle	x – 1, y	x, y	x + 1, y
Bottom	x – 1, y + 1	x, y + 1	x + 1, y + 1

I tried a few different ways around this, but I kept getting errors regarding the vertex buffer.

From what I understand, I am not using all of the vertexes in my indexes. I spent a great deal of time researching buffer geometry, Second Life sculpt maps, and anything regarding spherical meshes. Eventually I got a solid object to appear from out of nowhere…

What a feat. Looking at the code, its very simple. That’s part of life. Everyones code is often simple when you look at it, but it takes a great deal of research along with trial and error until you achieve your goals. There are multiple parts to this problem. Identifying that the sculpted prim images are in fact representative of a spherical object. This helps me determine if I should wrap the x/y coordinates, clamp them so they don’t go out of range, or just do the calculation without guards. What currently works is removing the guard.

function sphericalIndex(x, y) {
  return y * (width + 1) + x;
}

Another part of it was determining the proper size of the vertices. I was using the count of control vertices multiplied by three to create an array since the position expects each value of a vertex to have its own element. This wasn’t exactly rite. For spherical mapping, I needed to do some math on the width and height of the image to include vertices for a hidden row and column.

  const controlMeshGeometry = new THREE.BufferGeometry();
  const count = ((width + 1) * (height + 1)) * 3;
  const vertices = new Float32Array(count);
  controlVertices.forEach(({ x, y, z }, i) => {
    const offset = i * 3;
    vertices[offset] = x;
    vertices[offset + 1] = y;
    vertices[offset + 2] = z;
  });
  controlMeshGeometry.setAttribute(
    'position',
    new THREE.BufferAttribute(vertices, 3)
);

The faces were another issue to work out. I had to work out a way to avoid adding duplicate faces. There is also a specific order that faces are supposed to be added to meshes to indicate if it is facing away or into the model. This is why in some video games, when you clip the camera inside of an object, you can still see everyone around you, and you may see things like your characters eye balls and teeth floating in space. Given the nature of a sphere, I thought I had to skip the first row and column and perhaps had to tie all vertices of the top and bottom rows to the first pixel on the left. In the end, that wasn’t the case.

  var indexedTriangles = [];
  for(let x = 0; x < width; x++) {
    for(let y = 0; y < height; y++) {
      const centerIndex = sphericalIndex(x, y);
      const topIndex = sphericalIndex(x, y + 1);
      const leftIndex = sphericalIndex(x + 1, y);
      const topLeftIndex = sphericalIndex(x + 1, y + 1);
      
      if(topLeftIndex >= 0 && topIndex >= 0 && leftIndex >= 0) {
        // Add triangles in counter-clockwise order
        indexedTriangles.push(centerIndex, topIndex, leftIndex);
        indexedTriangles.push(topIndex, topLeftIndex, leftIndex);
      }
    }
  }
  controlMeshGeometry.setIndex(indexedTriangles);

Apparently everything isn’t perfect just yet. Some objects have jagged edges or disconnected layers. Others are not fully shaped correctly as they seem to bend into themselves. The comment box had a tapered base, and the table is more of a solid cube instead of being supported with individual posts.

Throwing Shade

I had added a light to the scene earlier, but it seemed that it wasn’t having any effect on the model. I needed to see some depth. Reviewing documentation, It seemed that light doesn’t affect the MeshBasicMaterial. I would need to use either the MeshStandardMaterial or MeshPhoneMaterial. Ambient light was working. I added a slider to the UI to adjust the ambient light, and found that it was responsive.

I needed depth, and ambient light wasn’t the answer. I focused on the directional light. Without ambient light, my model was black. I tried changing the lights position, target, and intensity. Nothing. I then looked into spotlights, and found the same issue. It dawned on me that it may be the model itself. I tried reversing the order in which I added triangles. Nope. I finally resorted to rendering a simple cube instead using BoxGeometry. I was able to make out three distinct sides. I got my depth.

Now comes the big questions. What is wrong with my models? Why does ambient light affect them, but directional light does not? Was this related to the jagged edges and odd interpretations of the models?

I started looking at one of the models that had jagged edges close up. I zoomed in and discovered that they weren’t actually jagged. They somehow were making many layers floating above each other. For some reason, the vertices along the edges were not connecting to vertex next to them in the 3D space. They seemed to be going into the center of the object instead.

It’s time to go back to basics. From what I recall, sculpted prims were small images (16×16 or 32×32). Some of my images seem to be a bit large, and others are not square. Second Life always stored the image size as being a base two. You could have a width or height of 2, 4, 8, 16, 32, 64, 128, 256, 512, 1024. The fee to upload an image was 10 Linden Dollars regardless of size. There was a tradeoff between how much detail you wanted your objects to display, and the time it took to load the object due to the size.

I displayed the height and width next to the selected image and found all heights and widths to be a power of base 2.

It looks like that’s a bit of a dead end. Perhaps not every pixel in the sculpt map is used. An image at 1024×1024 would potentially have 1,048,576 vertices and 6,291,456 triangles. Just one sculpted prim could potentially cause havoc for and player in range of the object. My region within second life alone could have 30,000 prims when these were first made available.

Looking at the Sculpted Prims Technical Details, it seems that sculpted prims were originally limited to 32×32, 64×64 and 128×128. 64×64 was highly recommended with a warning that 32×32 wasn’t officially supported. I’m assuming that 32×32 images was the smallest size that the Second Life viewer supported in general. For some reason I have a few sized as 16×256. The standards appear to have changed.

There is also mention that the poles were determined by width/2 on the first and last row of the texture. In my earlier attempt, I had assumed they were the left most pixel.

More analysis started to come to light with some sample code they provided to write a TGA file in C++. The comments indicated that odd numbered rows and columns are not used. The blue channel was set to 0.98 for pixels in the last column (for planes) as well as pixels in the last row (for planes and cylinders).

I changed the code around to skip odd numbered pixels, setup the poles for the first and last row, and stitched the left and right sides together. To my surprise, my jagged edges became strait. My table started to reveal its individual legs. Even my mess of business cards started showing individual rectangles.

The structure of the objects still had problems, but they were more recognizable now. Some of the objects such as the 45 arm To think that over 75% of the image isn’t used in retrieving vertices is a bit shocking.

I still had a problem with the object not receiving any directional light. It has to be something to do with my vertices, or the order that I create the faces. Reading Avalab’s The Basics of Sculpted Prims by Gaia, they mentioned the bottom left corner of the image a few times. I’m reading left to right, top to bottom. They also mentioned that a 64×64 sculpt map has exactly 1,089 vertices and 1,024 faces. A quick addition to display mesh details gave me 1,024 vertices and 2,112 faces.

So what’s going on? I’m missing 65 vertices. I suspect it has to do with the extra row and columns. Both the technical explanation on Second Life’s wiki, and Gaia’s chat log mention there are 33 rows and 33 columns, making 32×32 images unstable as sculpted prims. I thought I’d be smart about it and just connect the vertices in the last column to the vertices in the first column, rather than duplicating the vertices. I think that’s still fine, and its specific to how they implemented the logic on their end. The same goes for the poles – almost. In reality, I could reduce the vertices further and only use one of the vertices in the center of the top and bottom rows.

The face count is what concerns me the most. I’ve got almost double the number of faces that they expect. Since all faces on the top and bottom segments map to the center of the image, I can create just one face for the top and bottom vertexes. Unfortunately it only brought me down to 1,920.

Something didn’t seem rite. How can you have less faces then vertices, when each vertex usually connects with 6 faces? Going back through Gaias transcript, they mention that the surface is made with rectangular faces called quads. I’ve always thought that faces were only triangles. Now things make sense. If I double their quad faces of 1,024, I get 2,048 triangle faces. Originally I had more than that, and now I have less. I think this is an issue with how everyone implements the creation of the geometry.

I think my problem may be more focused on the order of vertices added for each face being in a clockwise or counter-clockwise pattern, or something to do with normals.

And… it was normals. There is a function on the BufferGeometry called computeVertexNormals that made everything appear once it was called. Objects immediately received the directional light and showed depth.

I found that my optimization to only include a triangle instead of quad on the top and bottom rows had a hole in the center of the object. Other things I’ve noticed is that there appears to be some clipping effects where a face flickers on the corners when the object or camera moves. The corners of objects also seem to have constant dark faces as well.

Wrap Up

I’ve spent the entire day on this little project. I haven’t gotten to do anything with NURBS modeling just yet. I did learn a few things though.

• Plot points in a 3D space
• Control the camera with a mouse
• Scale objects on the X, Y, and Z axis
• Create a mesh from vertices
• Create faces from vertices
• Construct faces from vertices in a counter-clockwise pattern to face out from the center of the object
• Add and control various lights on a scene
• Compute normals for an object to receive light
• Details regarding the Sculpted Prim format of images

Given that 75% of an image is not used with sculpted prims, I’m reconsidering if the format is even worth using. Images seem to be more web-friendly in that they probably wouldn’t cause issues with firewalls and are pretty to look at. I may just use the OBJ format or something similar after all. I want to finish getting the sculpted prims to render properly in my browser as well as display textures before I move on. I’m curious about NURBS as well – which was the goal of starting this little project.