Growing Up and Growing Large: Modern JavaScript Web Development Using Sails.js and AngularJS (Part 1 of 3)

The JavaScript Revolution

I will be the first to admit: had you told me that JavaScript would become the de-facto standard for web development a few years ago, I would've laughed in your face and requested that you take some breathalyzer measurements.  How could a language reserved for some ugly client-side DOM manipulation and the general clutter of the view-side of MVC architectures become anything close to enterprise-grade?  Heck, JavaScript was even an afterthought, built in 10 days by Brendan Eich to give browsers some whizbang on the client-side.

Well, the numbers don't lie: JavaScript development has grown up and grown large...but this ain't your mid-to-late 2000's JavaScript, folks.  We are talking highly scalable, testable, and standards-compliant multi-platform development.  We are talking a language that is very well suited for using NoSQL data stores.  We are talking a language that empowers developers to react and implement quickly.  Modern JavaScript is to past JavaScript as a jetliner is to the Wright Brothers plane.  It is time to take it seriously and for practitioners of classical languages to bone up on this quirky but highly effective interpreted language and its frameworks.

Lean and MEAN

PHP had the LAMP stack (Linux/Apache/MySQL/PHP).  JavaScript has the MEAN stack (MongoDB/Express/AngularJS/Node.js).  Like with the LAMP stack, each letter of the acronym represents important components.

"M": MongoDB

MongoDB is the data store of choice for JavaScript-based frameworks.  It is, indeed, a NoSQL database: a schemaless document store that, out of the gate, supports REST and JSON, as well as scale-out capabilities (sharding) that you can expect from a NoSQL platform.

"E": Express

Express is a Node.js-based framework (Node.js is, incidentally enough, also part of the overall stack) that represents server-side JavaScript.  Yes, server-side like ASP .Net MVC and Spring MVC.  This is the Server API portion of a MEAN stack application, the keeper of domain objects via RESTful endpoints.

Sails.js is an excellent MVC framework built on top of Express, and is the framework we will use for this thought exercise.  For .Net folks, think of Express as IIS and Sails.js as ASP .Net MVC.  For Java folks, think of Express as JBoss and Sails.js as Spring MVC.  You could technically build a "traditional" server-side web application on Express without the use of Sails, if you so desired.

"A": AngularJS

AngularJS is a client-side single page application (SPA) framework heavily sponsored by Google that aims to "extend" HTML to represent the dynamic nature of web applications and provide a slick, seamless user experience.

"N": Node.js

While technically a subset of the "E" part of MEAN, Node.js does so much more than just the server-side operations.  Node.js helps manage JavaScript library dependencies for the Server API as well as AngularJS (using NPM...think Maven for Java or NuGet for .Net).  It can also help streamline the "build" workflow of a MEAN stack application by running unit tests (in conjunction with a task runner, like Grunt or Gulp).  Node.js is essentially the glue that binds all the elements together.

A Typical MEAN Stack Web Application

A "typical" use case of a web application

As the diagram above indicates, MEAN stack applications give your typically nice, segregated separation of concerns.  

Sails.js on the server-side exposes RESTful endpoints for external applications to interact with the domain of the application.  The domain is persisted on MongoDB and you'll have "POJSO's" (Plain-old JavaScript Objects) for Sails.js to interact with the persistence layer.  In case you're wondering, yes, you could easily swap out MongoDB with MySQL or PostgreSQL if you're still unsure about NoSQL platforms...but I'd recommend against torturing yourself in that fashion.  ;)

AngularJS on the client-side focuses on a rich user experience, shuttling any of its data needs to the Sails.js Server API.  Likewise, if you were to build, say, a native Android or iOS application, they would interact directly with the Sails.js Server API to get at your application's domain object model.  True multi-platform development!

While the representation above isn't anything new in software, doing a full MEAN stack offers many benefits:

1. JavaScript end-to-end.  No context-switching for developers when they cross the boundary from client to server-side.  (Read: faster development, real collaboration between client and server)
2. Built to scale out.  AngularJS is a SPA loaded on the client.  Sails.js runs on Express and Node.js which was designed to scale out.  Likewise for MongoDB.
3. JavaScript has grown up.  This ain't your daddy's JavaScript.  Want complex collection or object manipulation?  npm install underscore (or lodash).  Need an ORM, Socket.io support, native RESTful/JSON support?  Comes out of the box with Sails.js and AngularJS.  Dependency Injection?  Built into AngularJS, and use require on Sails.js.  Unit testing and mocking: do you prefer jasmine or mocha?

Okay, So Let's Build Something!

In Part 2 of this 3 part series, we're going to get started with a bottom-up approach.  Using Sails.js and MongoDB, we will create a domain model and persistence layer for a movie application called the AgileMovieDB.

Orlando Code Camp 2013: SQL 2012 BI

Many thanks to everyone who attended my Orlando Code Camp 2013 session on SQL Server 2012 BI.  There is great potential for the Tabular Modeling of SSAS, and I hope you're excited about using it for your BI needs!

Here are a few links that I promised that will help you get started quickly in using the entire SQL Server 2012/SharePoint Server 2010/PowerPivot and PowerView stack:

• Install Process of SharePoint 2010 With PowerPivot and PowerView Support
• A Detailed Guide to Tabular Modeling for Beginners
• Quickstart Guide for PowerView
• Telerik's Reporting Controls: SSAS Tabular Visualization "A La Carte" (for those who prefer not to use SharePoint/PowerView)

For those with Subversion who want to get at the artifacts from the session yesterday, perform a checkout on https://edg.sourcerepo.com/edg/OrlandoCodeCamp2013 to get the Visual Studio solution and the backup of the NFLDW database we used as our source for SSAS Tabular.

Finally, feel free to contact me on Twitter (@grales) or e-mail (eric.v.nograles@gmail.com) if you have any questions/issues or wanted to bounce some ideas around SSAS tabular and its applications in corporate BI.

Thanks again to the Orlando .Net User Group for the opportunity to speak at this fun event!  Hopefully, I'll be seeing you all again next year!

The AgileThought family thanks everyone for attending our sessions at Orlando Code Camp 2013!

Packaging Existing SQLite Databases With Your Google Android Application

Many examples of Android applications on the Internet assume that the apps you develop will, by default, create a new, blank MySQL database at its first run-time.  However, there aren't many examples of a situation where one would create a separate MySQL database which would then be used by the Android application.  A potential solution, borrowed from databases on other platforms that use Continuous Integration, would be to embed seeding SQL statements in the application to run on creates or upgrades, but this may be prohibitive for some developers in terms of practicality and time, in addition to the fact that Android does some backend wizardry with SQLite databases to have them work properly with its SQLiteOpener class.  So, how would one embed an existing SQLite database to the application's assets folder and use it at runtime?  Stay tuned for the solution after the jump!

Using Unity for Dependency Injection With WCF Services

The Dependency Injection (DI) pattern of software development offers many benefits in the area of separation of concerns.  The loose-coupling nature of this pattern allows for truly atomic unit tests and (theoretically) more effective development.    

While the DI pattern is well documented in web UI technologies that espouse separation of concerns (such as MVC), the use of this pattern in the less glamorous area of application integration using web services is a little leaner on the volume of documentation.  Being that application integration apps such as WCF Services have a tendency to perform some elaborate transportation and transformation logic, having the benefits of the DI pattern greatly improves the effectiveness of the development of these applications.

So, in terms of WCF Services, how exactly do we achieve the DI pattern?  Thanks to the lightweight Unity library, we can offer the following DI benefits for a WCF Service:

• File-less activation of services (no more pesky .svc files to maintain)
• Loosely coupled development
• Rapid and agile development thanks to unit testing

Coding commences after the jump!

Business Intelligence Shootout: Microsoft SQL Server 2008 BI vs. Pentaho BI Enterprise Edition (Part 2: SQL Server Analysis Services)

The BI Platform That They Already Own
So, now that we've seen some of the basic capabilities of Pentaho Analysis Services (aka Mondrian), let's check the other side of the ring where SQL Server Analysis Services (SSAS) sits.  The entire SQL Server BI stack is a very interesting case.  Starting with SQL Server 2005, Microsoft began packaging their entire BI suite with a Standard Edition license, and I mean the whole shebang.  SSIS, SSAS, SSRS, SSMS, and BIDS...the whole gang to satisfy all data analytic needs.  Apparently, this was not emphasized enough in the literature for Microsoft SQL Server, because more often than not, this can be news to IT folks in enterprises.  Usually, it's good news, as the company may have already made a sizeable investment in SQL Server, and the icing on the cake is a world-class BI platform.

Now, for those companies who haven't already made an investment in SQL Server, the barrier to entry for SQL Server BI may be the price of a license.  After all, the bulk of the license fee pays for the RDBMS, one would argue.  However, this does not diminish the value of the SQL Server BI stack.  This platform has come a long way since Analysis Services was first revealed in SQL Server 2000, and we will take a deep dive into the capabilities of the latest features within SQL Server 2008 R2 Business Intelligence.

Business Intelligence Shootout: Microsoft SQL Server 2008 BI vs. Pentaho BI Enterprise Edition (Part 1: Pentaho Analysis Services)

A Tale of Two Analytic Engines

In this first head-to-head comparison, we pit Microsoft SQL Server Analysis Services (SSAS) 2008 against Pentaho BI's Analysis Services.  As mentioned in the introductory post, in-memory analytic engines aren't exactly new hat.  In fact, the inspiration for these engines came from very simple spreadsheet applications, hence the origins of Essbase's name -- "Extended Spread Sheet Database."  Despite their age, the benefits of analytic engines remain the same: data retrieval and ad-hoc analysis at lightning speeds.  Because most of the data is persisted in RAM, the speed at which you retrieve the data is only limited by your network speed and your user interface's rendering.  With that in mind, we have two analytic engines here that drew inspiration from similar roots, but go about their implementations differently.  

Microsoft SQL Server Analysis Services has actually been around since SQL Server 7, thanks to Big Redmond's acquisition of Panorama Software.  It has only been a recent development, starting with SQL Server 2005, that Microsoft has made a serious push into the BI space, literally offering its entire BI stack for free with a SQL Server 2005 (and then later, 2008) Standard Edition license.  Microsoft innovated the now ubiquitous Multi-Dimension Expression (MDX) query, and has made strides in usability, deeply integrating the SQL Server BI platform to all of its core enterprise offerings, Microsoft Office and Microsoft Sharepoint, as well as its well renowned integrated development environment, Visual Studio.

Pentaho Analysis Services, aka Mondrian in the open source world, is a relative newcomer to the BI marketplace.  Started by industry veterans from the defunct Arbor Software (where Essbase was incubated and released) at the turn of the 21st century, this Java-based analytic platform began its roots as an open source platform, along with the other components of Pentaho BI.  The goal of its founders was to create a powerful, flexible, cohesive, scalable, and cost-effective platform, meeting or exceeding the capabilities of its commercial conglomerate counterparts.  Who better to architect and develop such a solution but some of the very pioneers of the BI movement?  PAS is the core of the Pentaho BI stack, offering seemingly the same capabilities as other analytic engines in the market.


With the history of both analytic engines in mind, let's take a deep dive at both, using AdventureWorks as the star schema base.  For the impatient, I have published all the artifacts produced in this blog post on my source control system, which grants everyone read access.  If you have a Subversion client, point to https://edg.sourcerepo.com/edg/PentahoAdventureWorks

Business Intelligence Shootout: Microsoft SQL Server 2008 BI vs. Pentaho BI Enterprise Edition (Introduction)

Business Intelligence: A History in Review
Business Intelligence (BI) has become a hot topic in IT and business in recent years.  In the past, BI seemed more like a gimmick, with disconnected pieces of software made by various purveyors that seemingly required a significant upfront investment from both IT and business resources, from both a fiscal and training standpoint.


You might have a tremendous analytic engine from someone the likes of Arbor Software with their Essbase platform, but to effectively syphon the data out into a presentable, maintainable format, you'd also have to invest in some sort of enterprise reporting tool like Arcplan.  Furthermore, to grant users the ability to perform ad-hoc slice and dice analysis, you'd need an analytic software package pushed by a company like Hyperion.  The overall experience of BI, much like its roots, seemed disjointed, with no common vision, and as a result, was expensive to learn and maintain and did not impact the business world the way it had hoped.


The days of disjointed BI have thankfully past, and we are in an era of computing where these tools have become nearly as slick as their web and desktop enterprise application counterparts.  Speed of deployment, ease of use, and cost efficiency are the key elements in modern BI.  Users want their data faster, more flexible, and robust enough to truly gain that holy grail of turning business intelligence into business INSIGHT. 


The Geekstantialism BI Shootout!
In this eight-part series, we will bring two up-and-coming BI platforms front and center, and objectively review the strengths and weaknesses of each, as well as make some recommendations on their optimal effectiveness.


In one corner stands a slick platform backed by the billions of the Microsoft Corporation -- SQL Server 2008 BI.


In the other corner, a young upstart Java-based open source BI platform architected by some of the brightest minds in BI since its inception -- Pentaho Business Analytics.


The Parameters
In order to gain the best insight of the strengths and weaknesses of both platforms when stacked up against each other, I've put together the following parameters/requirements that both must follow in this thought exercise...

Pentaho System Setup

• Ubuntu Linux Server 11.10
• Apache Tomcat Server
• MySQL database backend

SQL Server 2008 Setup

• Windows Server 2008 Enterprise Edition
• IIS
• SQL Server 2008 backend

Data Source
The base data warehouse for both will be the AdventureWorksDW2008 database offered by Microsoft.  Obviously, as Pentaho BI runs off of MySQL out of the box, I will be converting said SQL Server database to MySQL, keeping it structurally the same.


Analytic Engine
Using the design tools for both platforms, cubes will be built against all fact tables in the AdventureWorksDW2008 database.  All dimensions will be created.  Dimensional hierarchies will be created where applicable.  Simple calculated members will be created.  Data mining options will be briefly evaluated.


Ad-hoc Analysis
The following ad-hoc slice, dice, and drillthrough will be performed against both platforms using the respective front-ends:

1. Investigation of Internet Sales by Product, Customer, and Sales Territory
2. Investigation of Reseller Sales by Order Date, Employee, and Promotion

Reporting
Both platforms will be evaluated in producing the following reports using provided design tools and persisting them to their respective report servers for distribution:

1. Call Center Report by Date Shift
2. Category of Items Purchased by Customer and Date.

Integration Engine
Both integration engines will be evaluated by using the design tools provided and will perform the following:

1. Refresh DimProduct with new data (CSV source)
2. Refresh DimCustomer with new data (CSV source)
3. Rebuild the respective cubes and dimensions on the respective platform

The Final Verdict
After going through the components of both BI platforms, I will outline the overall strengths and weaknesses of both, and make some educated recommendations on the optimal implementation of either platform.  The goal here isn't necessarily to determine which is truly "better" (as that can be a relative term, depending on the circumstances), but instead to gain a better understanding of what scenarios would call for what tool.  After all, that's the whole fun in technology!


My Qualifications
Finally, if you haven't done so already, you must be asking yourself what makes me qualified to make any sort of judgement call on BI platforms?  Professionally, business intelligence has been my specialty area for the past 6 years.  Having been involved as the lead developer for multiple BI platforms on projects of varying sizes in finance and investments, I have practical experience on BI implementations and the demands of its users.


I do have a depth of knowledge with the Microsoft product stack, as that has also been my professional specialty for about 9 years now (specifically Visual Studio [C#, VB, ASP .Net], SQL Server, and the SQL Server BI stack).


I also have a depth of knowledge in the open source and Java, always experimenting with various flavors of Linux (from RedHat to Mandrake to Debian and now Ubuntu).  For fun, I like to develop Google Android apps.  


This will be my first deep dive into the Pentaho platform.  I hope to gain and document insight into this well-regarded platform and see how it stacks up to the SQL Server BI stack that I know well.  Furthermore, I've always had a soft spot for open source initiatives, but found many times that they're a little too rough around the edges for risk-averse enterprises.  Pentaho looks a bit more promising.


Pentaho claims to be able to lure people away from "big commercial BI" with its platform, and I want to put that to the test.  Is it really a BI platform I can confidently recommend as a viable, and even superior, option when scoping out projects?  Time will tell, and I'm excited to find out!   




Stay tuned for Part 1: Pentaho Analysis Services!