t's profileThomasGoddard.comPhotosBlogNetworkMore Tools Help

Blog


    November 07

    Maya to XAML Goes Open Source!!

    Maya To XAML is now available to developers at large to help build its integration into Maya.  If you're a 3D graphics developer or just looking to learn more about Maya or XAML, the project is now accepting requests to join so get on over to CodePlex and sign-up!!

    Download Microsoft Expression Interactive Designer to easily view and edit the XAML exported from Maya.
    August 09

    Maya to XAML Updated for Maya 8

    An update for Maya to XAML is now available on Highend3D.com.  It has basic support for animation and should work fine with .NET Framework 3.0 July CTP.  See the ReadMe.htm file for detailed changes and specifics on animation.  Hope you enjoy and please let me know if you have any feature requests or issues.
    July 17

    The .Brain Framework

    A pioneer in mobile computing, co-founder of Palm (Graffiti), and intellectual conduit of the human brain for technology, Jeff Hawkins explains how memory works in the neocortex. He founded the company Numenta, and developed a memory model that works like the neocortex. The software architecture looks very solid and after reading the whitepaper and hearing his interview on NPR, I was compelled to share this with everyone.
    About a year ago, after sharing the book about this technology (OnIntelligence) at a dinner conversation with family friends, one of the people at the table unbenkownst to me Dr. Harry Saal, explained that he was a member on the board of directors for the company Numenta.
    The entire idea, that having a computer as good as a human brain, is a bit overwhelming and having made the connection with Haary, I feel like there has to be a future in this design. Take a look at the links and post your comments about it here. I am very interested in hearing what other people have to say about this new architecture.
    June 10

    WinFX gets a new name, .NET 3.0!!!

    It’s refreshing to know that I've been working on .NET 3.0 for the last year! It's amazing and I love the new name, which is now much easier to explain. If you think about it, the switch makes perfect sense because the CLR is solid and does not require much in terms of core language feature development anymore. By naming new extended libraries the ".NET Framework 3.0", developers and users get a clear picture on where we are in terms of the core API and the adoption rate goes through the roof. This is going to make life much simpler.

    If you havn't seen many of the cool features in WPF yet and would like to take an in depth look at a WPF application, check out this presentation from my colleague Darren David.


    Very wise move MS!!! Here is the original post sent to me from Craig Jaris and Karsten Januszewski, via Fluid's Chief Experience Officer seen in this interview

    March 03

    Maya to XAML February Update

    It's finally here!  I got around to creating a release build of Maya to XAML for the February WPF CTP.  It's not quite feature complete but it's getting there.  I added some helpful grouping features so that Maya groups are converted to a <ModelVisual3D> node and the Maya group's content is nested within the <ModelVisual3D.Content> tag.  Transforms are also applied to ModelVisual3D, as well as individual objects, so you get the inherited transform behavior with each transform in it's original state.
     
    The export options are updated so that you can specify export, or not, for normals, meshes, lights, cameras, etc...  I added lights and cameras to the exportable objects.  Animation is getting close to completion but I am swamped with higher priority features, like optimized mesh data.
     
    Keep a look out for a future release that includes animation, XAML formatting, and a few other helpful features.  Please let me know how I can improve on this plugin so that it's more useful as a tool for professional developers and designers.  Enjoy!
     
    February 06

    Starbucks Howard Shultz and the Hear Music Media Bar

     

    On the left is Starbucks CEO Howard Shultz.  I'm on the right.  The store to the right is the store that just opened in San Antonio.

     

    A man came in to the Starbucks on the Promenade in Santa Monica, CA, where I was working on the HP Media Bar. He was dressed in a pink oxford shirt and looked like a swanky business man. It was brought to my attention that it was The Man himself. I walked up to him and introduced myself as the Lead Solutions Architect from his partner company, HP, and explained that I had helped to get this store operating smoothly.To my surprise, he grabbed my hand in a hearty handshake and introduced himself as Howard Schultz, the CEO and Founder of Starbucks Coffee Company. He was interested to continue our conversation, get my thoughts on the project and ideas about the solution. He bought me a coffee and we began our conversation down the Promenade.
     

    In turn, he answered some of my questions, like how he got his start in creating one of the largest and most profitable Global corporations. He has written a book on the story and says that he had been able to gather capital for the first Starbucks from money he raised at the JCC and his basketball chums. Although his idea was a success, he continues to in turn lend funds and be a Philanthropist and friend to the Jewish Community. Being entrepreneurial myself, I see the value in the lessons he shared that day.

     

    The Hear Music Media Bar is a great vision that brings something new to retail.  The Miami store opens next week, according to sources at Starbucks ;-).  Look for more stores in places like New York.  I think solutions like this which use vector based UI with a web services backend will continue to be the trend in kiosk solutions.

    January 11

    Theeey're heeere... Mac & Intel

    I received an email from the apple Developer Connection today with some links on how to transition existing Mac OS X binaries to Intel based binaries.  This means a bit more work for developers to support Mac OS X non-Intel based platforms and intel based platform. 
     
    This also brings up a point, even though Mac is releasing a very large update to Mac OS X, they are not branding it as entirely new operating system.  This is a smart move from Apple, keeping the point clear that customers get the same look and feel in their applications, now with a faster and cheaper chipset.  Here is the resource center from Apple, with some interesting links on the new Intel Core Duo Processor and developing for the new Intel platform.

    http://developer.apple.com/transition/index.html

    I found this link to be pretty interesting:  http://www.intel.com/cd/ids/developer/asmo-na/eng/255716.htm

    Note:  The biggest difference in the core architecture is Endianess.
    December 29

    WinFX December CTP Changes

     
    BTW... I will post an updated version of Maya To XAML soon.  I am just wrapping up some new features so stay tuned!

    Thanks
    Karsten Januszewski and Michael Swanson
    December 19

    Visual Studio 2005 - Customer Excellence Award

    This makes it all worth while and it feels good to be recognized for working late nights to help improvide the product for everyone.  I look forward to working with the Visual Studio Team on future product releases.
     

    "You have been nominated to receive the Award for Customer Excellence. This award recognizes your extraordinary contribution to the Visual Studio 2005 product.

    All the best,
     

    S. Somasegar
    Corporate Vice President,
    Microsoft Developer Division"

    November 21

    Maya to XAML November CTP

    This update fixes some syntax in the XAML and has a couple minor changes to work with November CTP.  Keep a look out for an update that comes with animation export and more!  I havn't had a ton of time lately so I've had to work on it during my days off.  Let me know if you have any other features that you would like to see and I will try to push them into the final build.

    Download

    New Features

    - Updated for the November CTP WinFX Runtime

    Get the November CTP Get the November CTP
    The November CTP (Community Technology Preview) is the latest version of the WinFX technologies. Included in this version is the WinFX Runtime Components (RTC), Windows SDK and Visual Studio 2005 Extensions for WinFX. (Nov 18, Download details)

    - New lines used for mesh data

    - Added named meshes and groups

    October 17

    Exporting a Maya Model to XAML with Textures and Materials

    It was a very busy weekend.  I wrapped up the tutorial for exporting Maya meshes to XAML, with textures!  I resolved some bugs in the plugin, and posted an update here:
    http://www.highend3d.com/maya/downloads/tools/3d_converters/3782.html

    You can find the tutorial below at:
    http://www.highend3d.com/maya/tutorials/modeling/polygon/237.html
     
    Acknowlegements:  A big thank you goes out to Michael Aarsvold, an employee for EA games, for helping to fix a couple bugs in the Maya To XAML exporter and for providing this cool bug model in this tutorial.  Also, thanks to Brian Buchmann for helping me with some bugs in the Maya To XAML exporter.

    This article illustrates the Maya To XAML plug-in, emphasing successful export of model data with textures.  Please note that the plug-in is still in the development beta and still requires a small amount of cleanup before you have the ability to export model and textures.  Future releases will incorporate settings which will allow you to specify these options during the export.

    Why export Maya to XAML? This will allow for 3 dimensional interactive user interfaces.  It will not be long before all applications incorporate XAML in one aspect or another.  This tool provides the designer with a bridge to the developer.  

    The Original Model:
    The example below is a screenshot of the Character created in Maya.  The left screen is the inverse kinematics and mesh (polygon) data.  The right hand screen shows the skin of the object in open GL.


    Preparing for your Export
    There are a few parts to this model that will need attention before the export.  You will first need to modify your source model.  I recommend that you save a copy of your scene as the clean version used for exporting.  In the scene above, skeleton transforms of the skin must be baked.  Select the top level object group node, in this case "Bug", and choose the Skin > Detach Skin Options Button.  Choose the settings shown below and click detach:

    You can delete your models skeleton after you've baked the skin.

    You might notice from the picture above, that the right and left side of the bug's body are identical.  Instances of the left side objects were used for the right side of the body.  That is to say, the same polygon point data is used for the right side as is on the left.  The instances must be made into their own polygon points in order to be recognized.  To do this, select the instanced node, duplicate the node, and then delete the instance.  The Maya To XAML plugin does not currently capture instanced nodes, so only one side of the mesh will get exported, unless the instances are turned into their own mesh data, as mentioned above.

    Because the mesh instances were exact mirrors of their parent instance, their scale is -1 and their normals are flipped.  The scale needs to be frozen on both groups and objects.  To freeze the scale, select the groups and objects in the outliner and go to Modify > Freeze Transforms Options button.  Choose the settings shown below and click freeze transforms:


    After the transforms are frozen, you can either leave the "opposite" attribute checked for automatic normal flipping, which will have the Maya To XAML exporter automatically flip the normals.  This is located in the Shape Node>Render Stats>"opposite", or you can reverse the normals of the uninstanced, frozen mesh, by going to Modeling>Edit Polygons>Normals>Reverse.

    Before you export the model, you will need to go to Edit>Delete All By Type>History.  This ensures that no extra mesh objects or other data is exported.  Once you've deleted the history, you can choose "export all" from the file menu and specify XAML.  Exportable scene data is saved to the xaml file.

    To view the xaml file, you can try installing the
    WinFX SDK September CTP, which comes with a presentation host, for viewing xaml files directly in Internet Explorer.  Checkout the xaml version of the mesh below:


    As refinements are made to this plug-in, some of these tasks will be automatically handled by the exporter.  In the meantime, I thank you for testing my product and I hope you enjoy.  You can download the latest plugin at Highend3d.com Send any feedback or screenshots of cool stuff you've done or errors you've encountered.

    Glossary of Terms

    • Mesh - The goemetry and polygon data of the 3D model.  Each polygon contains a set of triangles and vertices.
    • Texture- In this tutorial, texture refers to the image or graphic applied to a material on the mesh surface.
    • Skin- The mesh of the model that is bound to a skeleton structure ( joints and bones ).
    • Bake- To make static and permanent or make a set of nodes that contain their own values into a single representation, with the previous nodes values making up the final result.
    • Polygon- A representation of a surface containing a set of vertices, triangles, texture coordinates, and normals.
    • Normal- The point of an edge, vertices, or surface that determines which direction in (x,y,z) it faces.
    • Scene- The space in which all data related to the model is represented.
    October 14

    Maya to XAML Exporter Updated (materials, textures, transforms)

     
    ** If you recently downloaded the update for Maya to XAML, I just released a new update again to resolve normal and model direction.

    Some Tips on Exporting:
    -Some mesh types need to be combined using Polygons>Combine, before exporting.
    -Removing groups will ensure that all of your objects are exported. 
    -You should delete unused textures and materials before exporting. 
    -Delete all by type history will also help to export the mesh properly.
     
    So it's been a week since I released Maya to XAML and I'm releasing an updated version already!  The plug-in is now capable of exporting materials, textures, and transforms(rotate,scale,translate).  I anticipate an updated release that includes animation in a couple weeks and a release before that, which will include lights and scene data.  I have a new download location for you as well:

    http://www.highend3d.com/maya/downloads/tools/3d_converters/3782.html

    I will also have a 3d character demo from a game developer at EA games.  Michael Aarsvold will be developing a rawkin character, with baked textures.  Check back on my site in a few days for the demo/tutorial.
    October 07

    Symbiotic Design: Maya To XAML Exporter

    Symbiotic Design:  Maya To XAML Exporter
    Using Alias | Maya with Microsoft Windows Presentation Foundation* aka Avalon

    **** Updated For Maya 6, 6.5, and 7.  Now includes textures, materials, and transforms!
    Download MayaToXAML Beta 2 
    If you receive an error opening the zip directly from the site or the file is empty or corrupted, try right clicking the link and saving the file to your desktop before opening.

    * this software is distributed freely, as is, with no warranties. 

    Installing Maya To XAML
    • Extract the contents of the compressed file.
    • Copy the correct version of the [MayaVersion]/MayaToXAML.mll file to your bin/plug-ins directory.
    • Copy the scripts/ExportXAML.mel to the Alias/Maya/scripts directory.
    • Launch Maya > and go to the Window Menu > Settings/Preferences > Plug-in Manager...
    • In the Plug-in Manager > Click loaded and auto load next to MayaToXAML.mll

    Using Maya To XAML

    • Open or create a new mesh file: File > New Scene or Open Scene...

    Note:  If your mesh is NURBS, you will need to convert it to polygons.  To convert to polygons, go to the Modeling view and select the mesh.  In the Edit NURBS menu, click the Rebuild Surface Options button icon.  Choose the Polygons option, in the Output Geometry section and click Rebuild.  You can optionally combine all meshes in your scene, before exporting.  This is suggested, for this version of the exporter, since models are not translated to new XAML scene locations during export.  However if you need to apply materials to each object separately, you can translate the models in Visual Studio .net after exporting.

    To export the mesh data, go to the File menu and click the Export All Options icon.  Choose the XAML file type.  The only export items supported in this version are objects, object instances.  Click the Export All button and enter the file name.  The mesh file is exported as a .xaml file to the specified location.

    Note:  You must place the scene in a new XAML window in order to use it in a xaml application.

    Here are some tests I ran for the exporter.  I ran the export on an 18mb mesh and loaded it up in Visual Studio .net 2005.

    Here is the Maya view of the model.  A huge thank you to Mike Fudge, an awesome 3d artist for EA Games.  His works include The Matrix, NHL 2005, and more!


    Click to enlarge 

    Click to enlarge

    Click to enlarge
    The final export output in a XAML window with a single light and camera.  MayaToXAML applies a default light, default materials, and a default camera.

    Click to enlarge

    Click to enlarge

    New Features

    • Materials, textures, transformations(rotate,translate,scale).

    Future Features

    • Lights, Cameras, Action!(animation).
    • Tutorial on game character with baked textures.
    • Real time XAML preview in Maya.
    • Mac OS X Version.  This is entirely unmanaged C++, so it should be a smooth port.

    Known Issues

    • Scene cameras and lights are not exported. A default camera and set of lights are exported instead.
    • Scenes with a very large landscape will have culling issues. To resolve this issue, turn the farplane distance up in the exported camera object.
    • Scenes with some group types do not export. Removing the groups will resolve this issue.
    • Export options do not actually do anything at the moment. All exportable scene data is exported.

    A Quick History of Maya and XAML/WPF
    Incorporation of 3D design in video production, computer generated animation, and graphics has created a sea change in the entertainment and media industry with the evolution of faster computers and more accessible broadband information access.  At the advent of designing in 3D in the digital space, people could envision the advantages and features that this type of design could have in desktop applications and the internet.  However it was difficult and time consuming to implement.  Resultingly it took some time for this technology to get a foothold in the market.

    In the mid to late 1990's companies such as Silicon Graphics (SGI) first explored the possibilities of 3D to enhance the user experience.  SGI broke ground with the first 3D chat and web player called "Cosmo Player".  This application was comprised of basic 3D modeling tools that could create realistic environments where people could meet online in a virtual space.  They could assign themselves a selected Avatar and put that object into an Open GL rendered scene along with other individuals in the same virtual space.  This scenario has become more practical and realistic in more recent times with faster processors and broader technology access.

    There were detractors to this setup.  SGI hardware was costly and they realized that they needed to stay on top of their competitors like Max and Lightwave on the Windows platform. Competitors, such as  Alias, released a version of Maya for both Mac and Windows.  Like Apple Mac OS of today, Alias Wavefront was able to develop the software to run on a cheaper Intel chip.  The goal to have entirely real-time photorealistic 3D on desktops by 2010 had been set by most software companies in the graphics sector. 2010- what a revolutionary year that could be when the best in graphics is available to the common PC not just industrial computers.

    SGI's commercial assets were Softimage and Alias, software applications.  SGI announced a spin off of Alias | Wavefront into a separate company and Microsoft purchased Softimage.  They both believed there was a future lay in developing these applications.  Meanwhile, SGI banked their future on high end hardware and the Irix OS that is used in medical fields and large government organizations.

    Another player in this field was an upstart company based in San Francisco, Macromedia.  They developed a vector based UI called Flash.  Their attempts at creating a desktop application platform were not as successful.  They ran a quick launch with Macromedia Flex but ran into cost issues and Flashes downfall being no windows integration and no propagation into the offline user experience.  This tool was not as attractive to desktop developers because it lacked the integration with core operating system functionality and the desktop development tools.  Arguably, they tried to fix these shortcomings with jGenerator, later known as Lazlo, and Flex.  Flash was developed as a web platform and pushed into the desktop and kiosk markets.  They looked at it from the top down instead of bottom up and pushed a web product as a desktop development tool, vs. a desktop development tool as a web product.  Flash will remain a widely popular platform for the web and will always be the cross platform leader in vector UI.

    Maya went from IRIX (SGI) to running on Windows (the same is true for Softimage), and Alias was just acquired by Autodesk.  Alias won an academy award for Mental Ray, "a highly programmable computer-graphics renderer incorporating ray tracing and global illumination to realistically simulate the behavior of light in computer-generated imagery." Alias|Wavefront's Maya received highest honor the Academy can bestow - an Oscar® statuette for "... the development of a 3D animation, dynamics, modeling, and rendering production tool known as Maya.  With its significant and dominant impact on the motion picture industry, the Maya software package offers a robust and widely used commercial visual effects tool with a rich infrastructure for extension and customization.",  so photo realistic that it has changed special effects in the motion picture industry.  Download a free version of Maya Personal Learning Edition at www.alias.com.

    Better graphics is the singlemost important feature of a compelling UI, and upstart companies such as Wavefront and Apple realized this.  Enter a surprising player in this field, Microsoft.  Microsoft applications have not been known for their compelling user experience.  Windows is a form meets function application, having none of the aqueous design like Mac OS X, for example.  The dark horse in this race, Microsoft made a non- headline-making purchase of Softimage.  Was this a peek into what they hoped to add to their user experience?  I can only imagine that Microsoft's rationale for this was to fill the gaps with Avalon, their next highly anticipated Windows Presentation Foundation and Windows Forms predecessor.

    September 14

    Starbucks HearMusic expanding coffee houses

    Back when I worked for HP Media Solutions - Consulting & Integration, I took part in the solution architecture and design of the Starbucks Hear Music media bar.  It was an awesome project to be a part of.  It's great to see it expanding into new markets in Miami and elsewhere.  We don't have one here in Palo Alto yet but some day, I hope...

    >Click here to see it expanding...

    TV 2.0

    I don't know if any of you have heard the buzz about TV 2.0 these days and where TV is going but I am sure we all have a good idea.  With my last post about sparkle, 3d, media, and programming you can infer many things about where the rich digital media platform is going.  Everything from your photos to your favorite TV shows will all be available anytime and anywhere.  But what is even more important to the user is how that information is presented and how easily it interoperates between applications.

    So have you heard of Microsoft IPTV?  You might want to take a look at it... It's the next generation of your TV/cable/internet/phone.  It's a single, unified entry point, to everything from buddies all the way to TV shows.

    I have an extensive video editing background working with Media 100 on Apple and Strata 3D before it hit Windows.  These tools combined with the detail of programming in C#, will give developers the abiliity to create content that reacts to user interaction and compells the user with every click.  It's crucial to realize the potential of this new toolset.  By embracing these technologies it no longer becomes a question of how well you develop applications but more about how much your users 'feel' your applications.  Hence the word expression in this new product suite.

    Sparkle on the way

    The days of flash with a web services backend are over.  The future is here.  I remember back in the day when we created macromedia flash UIs like realtycomposer.com or club90.com and I had to convince everyone that the best implementation for the application is a web service. 
     
    Using vector was the bleeding edge, using .net was the bleeding edge.  Now they've come together, as I knew they would.  It's now more important than ever to maintain a single application schema across all tiers.  By maintaining a single schema, each developer gains the type definitions without having to rethink the object.  Using WinFS as the database, the object store, and the business layer, you can call different resources on demand and present them in a rich 3d UI.
     
    It combines the workflow of adobe after effects, macromedia flash, and visual studio.net...  It's the release of WinFS and WinFX that are going to change the way we develop our applications.  But don't get me wrong, the applications built on top of these two technologies are already shaping the playing field and Sparkle IMO, is at the top.

    If you have not seen it in action, you gotta check this out!  You can skip past the chatting about 5 or 10 minutes in.
     
     
    And if you want to see an app built in WinFX - http://www.microsoft.com/max/  install it and I'll share some photos.

    If this doesn't shock you, I don't know what will...

     
    September 10

    Live from KeenMedia.net

    Here with the owner of KeenMedia.net and Sundial Music...  Aaron Granick (KeenMedia.net), Aaron Mojo (SundialMusic.com), and Oshkar Pakash (Ohskar.com)...  They finally finished their new studio which is pretty sweet!  Lots of keen things and kewl stuff.  We are spending this weekend working on business vision scope and brainstorming before Granick and Ohskar branch out in San Diego.

    Aaron Mojo gives us a preview of the new theme song, he wrote and sang, for a cartoon show on the WB called Johnny Test and the upcoming release of the game Shrek Super Smash up...

    Take a look at the live action studio:
    Download Movie


    You remind me
    September 09

    My Handy Dandy SQL Notes

    These are some notes that I took while studying for my MCSD final exam on SQL server 2000.  There are some good things in here and some things that might not make sense until I organize them and put them into a nice little paper.
     
    Linked Servers
    • sp_linkedservers lists linked servers
    • If you want different passwords between linked servers, you have to map the usernames by creating a user account on each server and then calling sp_addlinkedsrvlogin (nameofserver, currentcredentials, localloginname, remoteloginname, loginpassword on remote server).
    • Use fully qualified name of remote server to query.
    • Must drop username links before dropping linked servers.
    Views
    WITH CHECK OPTION for Views
    Prevents users accessing the view from editing the values in such a way that the data would not get selected by the view.

    INDEXED VIEWS
    Allow data that is retrieved frequently to be retrieved faster and must use ansii nulls option on.  A clustered index must be created first.

    UNION
    Should be used to join two tables in a view from different disparate databases and order by must be specified outside of view.

    COALESCE selects one or the other
    To compile queries remotely use the OPENQUERY function, only to be used with linked servers.
     
    OPENDATASOURCE in FROM can be used to access non-linked remote servers in T-SQL by passing all params when the function is called.
     
    OPENROWSET is used just like OPENDATASOURCE to query data remotely.

    Complex Updates
    • UPDATE T1 SET Column2 = 25, Column3 = 250, WHERE Column1 = 2
    • Must specify table names in updates with ambigous columns from 2 tables or table in from or where clause.
    • After aliasing a table name you cannot use the table name.
    • You can specify table aliases in FROM statement

    Inserts

    • If you use a different number of columns for an insert than the table contains you must specify the columns.
    • INSERT Table1 SELECT * FROM Table2 only works for direct column type mappings or int to character.  You can also optionally specify columns to insert from destination and source.
    • INSERT INTO with SELECT only used for small queries Simple Query
    • Use BCP for large text file imports.  Export from source to text, insert to destination from text.  Cannot delete rows if they are used in foreign key relationships.
    **DELETE WITH JOIN EXAMPLE**
    Delete od
    FROM [Order Details[ od JOIN Orders o
    ON od.OrderID = o.OrderID WHERE DATEPAET (yy, OrderDate) =1996
     
    Cursors
    • Forward only, fast forward only, Scrollable, Dynamic, Sensitive
    • Changes made outside the cursor are visible
    • Supports positioned updates and deletions, unless  defined as read-only

    Concurrency Control

    • Read-only cursors do not lock the rows from outside changes
    • Optimistic checks if a row is modified (important not to lock data but only one transaction can succeed)
    • Client refreshes the cursor if cursor is modified

    (TWO METHODS FOR CHECKING CONCURRENCY) 

    • Row Versioning, timestamp column, columns must include the rowversion data type
    • With Values, checks the row checksum

    Pessemistic (guaranteed transaction) ... All changes will succeed

    • Cursor Implementations
    • Client Cursors (Default rowset is returned to client)
    • Server Cursors (T-SQL or API based)

    Database APIs

    • OLE DB
    • ODBC
    • DB-LIBARY - Legacy only supports up to 6.5, Still support SQL 7.0 but can not use new SQL server features.
    •  ADO - High level API used with any programming language that supports Active X.  Encapsulates OLE DB and ODBC.    Limits some of their inherited functionality.
    • Client Cursors (Cached Rows on client)
      • Implemented internally by ODBC, DB-Library, or ADO.
      • Are not available in T-SQL- DB returns full resultset for each query
      • Increased memory requirements on clients
      • Supports forward only and static, NOT Dynamic
      • Support all select statements
    • Server Cursors
      • Can be defined by DECLARE CURSOR statements (can fetch only one row at a time)
      • Can be defined by using DB APIs (can fetch multiple rows at a time)
      • No multiple rowsets
      • Do not support COMPUTE, COMPUTE BY, FOR BROWSE, INTO
      • Do NOT mix T-SQL (DECLARE CURSOR) and API cursors (DB-Libarary, ODBC, OLE DB)
      • OPEN cursor always starts before first row.  Must use FETCH NEXT to start at the first
      • OPEN cursor is default forward only.
      • Free cursor resources using DEALLOCATE [cursorname]
      • DECLARE [cursorname] CURSOR SCROLL to declare a scrollable cursor
      • FETCH to retrieve rows in specific positions ABSOLUTE
      • FETCH relative grabs a row relative to the cursor's current position
      • DELETE [cursorname] WHERE CURRENT OF [cursorname]

    When to use default resultset vs. cursors

    • when using full resultset
    • when it is not necessary to visually analyze data in order to target specific rows
    • use client cursor to scroll through large result sets (static forward only, support all select statements)
    • use server cursor to perform positioned updates after visually analyzing the returned rows
    • use API cursors to fetch multiple rows
    • dynamic cursors open faster
    • keyset-driven and static cursors usually work faster in joins
    • use optimistic concurrency control to maintain high concurrency
    • use pessimistic concurrency control to ensure that each attempted update succeeds

    Working with XML
     Note:  FOR XML must be added to select statement
     RAW

    • each row is presented as an element with 'row' as the identifier
    • tabular
    • rows appear in the same order as the select list
    • all elements are at the same level
    • IE requires that there is a top level element
    • you can specify the root node as part of the param using root word

    AUTO

    • Each table is represented by an element
    • corresponding attributes are columns
    • tables are nested in the order in which they are selected with the columns listed
    • specify with elements so that the nodes are in elements rather than attributes

    EXPLICIT

    • use this to specify the format of the XML in the query
    • use column names and attribute names in the query in order to markup the schema
    • specify element in the column to specify element based output in XML rather than attribute based output
    • used for full control of XML outputs

    Extracting relational data

    • OPENXML
         DECLARE @idoc int, @doc nvarchar (1000)
         **EXEC sp_xml_preparedocument [@docout] OUTPUT, [@docinput]
       SELECT * FROM OPENXML (@idoc, 'Root/Customers')
    • Schemas must be matching to specify schema using WITH [schemaname]
    • Remove the internal document sp_xml_removedocument
    • schemas on the fly must specify each node after WITH statement ([columnname int '../@xmlattributename]), <<xpath
    • XPATH is case sensitive
    • flags are used for element centric data, must specify schemas or else values will be null

    Programming Business Logic Transactions

    • A logically complete unit of work
    • all data in logically consistent state
    • changes to data must be permanent during system failures
    • Concurrent transaction are isolated from others
    • Programmers are responsible for the logical consistency
    • SQL Server is responsible for the physical integrity of data

    Transaction Modes

    • Autocommit - each T-SQL statement is a separate transaction
    • compile time errors do not build transaction plans but table"runtime"errors do execute
    • Explicit - a user explicitly starts and ends transactions (auto rollbacks on failure) constraint violations must   correctly be rolled back by users
    • must specify rollback when failure occurs and all changes will be rolled back.
    • the only reason SQL will rollback automatically is if a connection failure has occured
    • if commit is called even if the transaction threw an exception, then data will be corrupted
    • Implicit will automatically rollback

    > SET IMPLICIT_TRANSACTIONs ON will prompt you to commit transaction when disconnect occurs
     
     Isolation Levels (ordered by increasing isolation)

    • READ UNCOMMITTED - allows dirty, or uncommitted, data to be read
    • READ COMMITTED - allows only committed data to be read (locks data from being read)
    • REPEATABLE READ - prevents other users from updating or deleting selected data
    • SERIALIZABLE - prevents other users from making any changes to selectable, reduced concurrency, must release locks

     Deadlocks

    • Occurs when one transaction is attempting to change data when another transaction has already performed a lock.
    • Error 1205 will occur if they both lock the data.  The second transaction is the deadlock victim
    • Deadlock is lifted when first transaction finishes
     Efficient Transactions
    • Keep transactions short
    • Avoid user interaction within a transaction
    • Open a transaction only to modify data
    • Minimize the use of implicit transaction mode
    • Minimize the amount of data targeted by a transaction
    • Use lower transaction isolation levels (improve concurrency)
    • When using cursors, specify optimistic concurrency control
    • Design transactions to occur in the same order as the data
     Error handling
    • check for errors after each statement but use RETURN to stop processing
    • @@ERROR returns the last error, it returns 0 if an error does not occur
    • BEGIN and END are control statements

    Stored Procedures

    • Input parameters
    • use @paramatername datatype and use comma to separate.  Up to 2100 parameters can be used
    • AS comes after parameter list
    • Output paramaters
     Triggers
     Special type of stored procedures
      > Trigger Mechanics
    • When INSERT, UPDATE and DELETE statements can fire triggers
    • INSTEAD OF triggers fire instead of triggering statements and before contraints
    • Only one INSTEAD of trigger can be created for each type of statement
    • AFTER triggers fire after triggering statements
    • Multiple AFTER triggers of the same type can be created on the same table
    • Triggers are event-driven stored procedures;  they are called automatically when specified data modifications are attempted
    • Triggers provide functionality that goes beyond that of constraints
    • Instead of triggers are executed instead of
    • After triggers are fired after

    Tuning and Optimizing Data Access
     > Methods for optimizing query performance

    • Application Design
    • Minimize network usage (batches)
    • only return required rows and columns SET NOCOUNT ON to eliminate the number of rows count
    • Cursors require more round trips. Limit cursor usage
    • Use stored procedures and avoid ad hoc queiries (send only params)
    • Move logic to data
    • Allow applications to cancel their queries (no reboots)
    • Follow the recommended practices for minimizing deadlocks
    • Keep queries and transactions as short as possible
    • Always specify time-out periods

    SQL Profiler

    • Is a graphical SQL Server monitoring tool
    • Capture information about events
    • Save to file or table
    • Replay trace events at normal speed or one at a time
    • Trace information can be used to identify and address performance issues and other problems:
    • Debugging stored procedures
    • Identifying long-running queries
    • Auditing user access to instances of SQL server
    • Use filters to specify certain aspects to look for
    • Create templates for your own custom traces

     Index Tuning Wizard

    • Tune custom selects or profiler results
    • Schedule and save the scripts for index tuning
    • Can apply changes now or later
    • Can view or Edit the tuning wizard script

    Statistics

    • Provide statistical information about the distribution of values in a column
    • Are automatically created on all indexed columns at the time of index creation
    • Can be created automatically or manually on non-indexed columns
    • Are used by the query processor to determine optimal data access strategies for query execution
    • Can be updated automatically or manually
    • Updating statistics is important but we should disable automatic statistics during business hours
    • (sp_updatestats

     > Most optimization happens at the client
     > Optimize at design time
     > Use stored procs in client apps
     > Use SQL profiler to debug and optimize stored procedures and batches used in application (locking, deadlocking, security violations)
     > Use the Index Tuning wizard to modify indexes in order to optimize a specified set of queries
     > Statistics on indexed columns are created automatically.Statistics need to be up-to-date

     

    Using Indexes to Optimize Data Access

    • Non clustered indexes are sorted in DESC order
    • B Tree is the start of a subindex
    • Each subrange is divided into two more subranges and 4 3rd level nodes.
    • This occurs until only one pointer exists pointing to the location on the hard drive where that row resides
    • In a clustered index the table index starts at the top and works its way down
    • Built in the same manner as the B-Tree
    • Clustered index are used as pointers to rows to a non clustered index

     > Fill factor

    • Page splits cause index fragmentation
    • Drop and recreate indexes
    • Specify the fill factor to specify how much data the table will initially require

    > Indexing strategies

    • Indexes on tables with many rows are better than small tables
    • Indexes on columns with unique values are more efficient than indexes on columns with many duplicate values
    • A clustered unique index is automatically created on primary key in each table
    • Too many indexes adversely affect the performance of INSERT, UPDATE and DELETE statements
    • A large number of indexes may improve the performance of select statements
    • A single composite index on multiple columns is more efficient than a separate index on each of those columns
    • A clustered index should be as narrow as possible
    • Clustered indexes are especially efficient on columns that have to be sorted or searched for ranges of values
    • Always list columns in appropriate order when using composite indexes
    • create indexes to find WHERE clause or JOIN condition data
    • Create clustered indexes on columns frequently search for reange of values
    • Create non-clustered indexes on columns frequently searched for exact matches
    • Create composite covering indexes for columns all contained in the select (greatly improves performance)
    • In composite indexes, list columns with higher cardinality before columns with lower cardinality
    • When creating indexes, specify appropriate fill factors to minimize fragmentation
    • Drop and rebuild indexes to eliminate existing fragmentation
    • Periodically rebuild indexes and update statistics to improve performance

    Designing database security
     > Providing Direct Access to Data

    Logins

    • Provides access to particular instance of SQL Server
    • Standard logins can be used from within third-party operating systems
    • Windows-based logins use integrated windows authentication

    Configuring logins

    • sp_addlogin [loginname], [password] to add logins SQL authentication (3rd party OSs)
    • sp_grantlogin [domain\loginname] for granting a domain user account to SQL
    • sp_grantlogin [domain\groupname] for granting a domain group

    Server Roles

    • Include logins
    • Only use existing roles
    • Roles specify tasks that the user can do

    Database User Accounts

    • Are specific to a particular database
    • Are mapped to databases
    • sp_grantdbaccess [loginname], [dbuseraccountname] in context connection of the db
    • each database must have a user for the login that is trying to access it and must be mapped

    sp_grantdbaccess

    • the username does not have to be the same as the loginname
    • to remove the linked access use sp_revokedbaccess [username]
    • mapping windows mapped user accounts is different format than sql auth sp_grantdbaccess 'Demo\Anne'

    Database Roles

    • Are similar to windows user groups
    • Are specific to a particular database
    • can include user and group accounts
    • Are fixed and cannot be deleted or changed
    • public is similar to everyone role
    • implicitly provide users with specific permissions
    • sp_addrole [rolename]
    • sp_addrolemember [rolemembername]
       

    Managing Permissions

    • Statement Permissions control statement level permissions
    • Are assigned in a particular database
    • Control the ability of users to create databases
    • Database specific
    • Assigned to roles and users of the DB
    • (Grant, Deny, Revoke)
    • GRANT CREATE TABLE TO [username]
    • DENY CREATE VIEW TO [username]
    • REVOKE CREATE TABLE TO [username]
    • REVOKE [permissionname] FROM|TO [username] use FROM and TO interchangeably
    • GRANT ALL TO [username] to grant all permissions in one step
    • REVOKE ALL FROM [username]
    • CREATE DATABASE permission is only controlled in the master database of SQL server
    • Object permissions control ability to modify data in objects
    • Can be assigned in a specific DB for specific objects
    • (SELECT, INSERT, UPDATE, DELETE, EXEC, DRI (declare referential integrity))
    • Can also be specified on individual columns
    • Implied permissions are acquired through membership in fixed server or database roles and through object ownership
    • May effective at the server, database or object levels
    • May not be transferable
    • (only admins or owners can delete or alter objects use dbddl for restricted authority for altering)

    Effective permissions

    • All non-conflicting permissions are combined
    • Denied permissions override the corresponding permissions
    • Last set of permissions overwrites earlier assigned permissions
    • All permissions are assigned at column level but table level permissions overwrite each column level permissions

    Roles permissions take precedence over user level permissions

    Designing Security Solutions

    • Application Roles (application security)
    • Are specific to a particular database
    • Can be used only by applications
    • Program applications to use application roles rather than users
    • Provide access from the security context
    • sp_addapprole [rolename], [password]
    • GRANT SELECT ON [tablename] to [rolename]
    • sp_setapprole [rolename], [password] will set the role of the user to the specified role
    • grant access to guest account for databases
    • Microsoft excel uses application roles

    Ownership Chains

    • Owns a set of objects that form a chain of reference
    • sp_addsrvrolemember [username], [rolename]
    • sp_changeobjectowner [objectname], [username] basically drops the object, clears permissions, and adds the user as the owner
    • if dbo and the current user are not the object owner then you must qualify the object name
    • Using Views, Functions, and procedures as security
    • You can query tables below views as long as the users are the same or the user access has access to both objects (the same rule applies to sprocs)
    • Users can only perform activities in the application roles, views, and procedures.

    Security Solutions

    • Can not restrict access to rowsets but can restrict access to views and revoke underlying table access from public.

    Security Solutions2

    • Use table valued functions to  update data in underlying tables and restrict access to a specific rowset and the current user with USER_NAME in subselect

    Binary trees of life

    In the adventure of object oriented programming, we often find ourselves in situations like the famous character Neo, from my favorite movie "The Matrix".  You are working in your everyday life, oblivious to the things around you and how they actually work, and then you find some tidbit that really makes you think.  It connects the puzzle and how oddly it connects, too.

    There are two major players in the world today... data and logic.  It's how we represent our data that really makes it workable in ways we never imagined.  Its the rules that connect the data that make it do the things, represent the things, make up the software, connect the systems, and bind the data into useful representations.

    This post is an exploration on the connection of data, the best way to represent data, and the way I perceive data.

    Lets jump right in and take a look at a simple example of a binary tree:


    In the picture above, the root node of the tree is four.  Four could corrispond to a specific object or memory address in our application. In fact, all of the numbers in the tree could represent keys with pointers to the actual records or data persisted in storage. A binary search tree provides an algorithm and dataset organized in such a way, that it splits the time it takes to find a specific element in the tree in half, for each traversal of a node in the tree.  This is also referred to as log2(n) in Big O notation, or sublinear because the running time is less than linear running time.

    The binary tree above is a balanced binary search tree.  Neither the left or the right side of the number four, root node, are ever 1 less or greater than the other.  That is to say, if we were to add a nine node to the right child of node eight and another right child of node nine, the balance of the binary tree would be lost.  The less balanced the binary tree, the less optimal the running time becomes.  At worst, the running time for the binary search tree becomes linear.  Linear running times are found in unsorted arrays, unsorted linked list (searches, inserts, delets), or contiguous data that requires CRUD functionality.

    We find this kind of efficiency useful for reducing the running times in graphics applications and games.  In a game, each frame must perform a set of events, or check a set of values against objects that exist in the current frame or current level of the game.  If you have ever done graphics development, you know that as the number of objects in the scene or canvas grows, the running time of each frame is increased.  Binary search trees are also helpful in determining which objects in the scene are culled.

    Check back as I continue my small series on binary search trees and provide credit to the amazing minds behind it.
    September 05

    Perl:: Get input from a command line

    It's a trivial task in C# but in Perl you have to understand what's required and what to call to get the input.  After you understand what's required, it's pretty simple.

    #< -- thats a comment
    #We have to use Win32::Console instead of Term::ReadLine
    #Term::ReadLine has issues on windows systems and will fail with "SetConsoleMode failed in ReadKey"

    use Win32::Console;

    #this is used to take hash values and convert to standard strings.
    use Data::Dumper;
    use Time::HiRes qw(gettimeofday tv_interval);
    use strict;
    use warnings;

    #First we set the terminal variable to the new win32 console.
    my $term = Win32::Console->new('Geo::Coder::US');

    #Now we allocate the memory for the terminal
    $term->Alloc();
    #Now we show the display
    $term->Display();

    #This will output some text to the terminal
    print "Enter some data below\n";

    #This is where the variable cool is set to the input of the user.
    #The screen will wait for input
    my $cool = <STDIN>;
    #This will clear any new lines in the cool variable
    chomp($cool);

    #That's it we can print the variable.

    print $cool