Professionals who believe that business intelligence is simply another buzz phrase thrown around by database experts are often left at the bottom of the totem pole of corporate production. These naysayers often perceive analytics tools to be an extraneous expense, but the technology is in fact becoming a necessity for corporations intent on surviving in an increasingly competitive market.
Reducing the amount of transfers
According to Campus Technology, Valdosta State University in Georgia recently improved its overall student retention rate thanks to business intelligence applications provided by database heavyweight Oracle. Before implementing the solutions, the institution retained a mere 67 percent of its first-year student body, costing the University an estimated $6.5 million in annual revenue.
With the assistance of a database administration service, the organization began integrating the analytics tools in April 2012 in a two-part transition procedure:
- In the first phase, VSU implemented Oracle Business Intelligence Enterprise Edition, a program possessing interactive dashboards, ad hoc queries and strategy management.
- The second stage occurred in the fall of 2012, in which the university launched Oracle Endeca Information Discovery. The software enabled administrators to collect and scrutinize student data from various sources.
After correlating the information gathered from Enterprise Edition and Information Discovery, VSU database administration noticed that pupils who eat breakfast on campus have a 10 percent higher retention rate than the majority, while freshmen who worked on campus had a 30 percent greater chance of staying at the school. As a result, the institution promoted on-campus eateries and invested $200,000 in student employment. A year later, VSU kept 2.5 percent more students than it did in previous years.
Interest is increasing
Interest in the technology has increased over the years, especially among companies in the retail industry. TechNavio reported that the global business intelligence market in the merchandising economy is anticipated to grow at a compound annual growth rate of 9.19 percent over the next four years. Due to the incredible volume of data retained by commodity-based businesses, remote database support providers are introducing more complex data processing tools to their systems.
"Basically, BI means getting the right information to the right people at the right time to support better decision-making and gain competitive advantage," TechNavio noted.
Universities across the U.S. strive to make their institutions more appealing than their rivals. This could mean lowering tuition rates, promoting certain curricula or renovating dorms. However, school administrators could get a better idea of where to allocate resources after consulting the algorithmic conclusions of BI.
Contemporary data warehouses are going beyond the basic store-and-save capabilities IT departments are used to seeing. However, due to increased usage of data collection and analysis tools, database administration services have managed more complex infrastructures that are better able to host these programs. An increase in server usage and action diversity has created an intricate environment demanding more assiduous maintenance and surveillance than was previously necessary.
The next big thing?
The organization's push for JSON is rooted in the prevalence of the Internet of Things. Companies are now using smart devices to amass millions of data points derived from the unique perspectives of each mechanism. The database experts claimed that the open standard format can offer organizations the agility needed to remain competitive. Business models, marketing campaigns and project developments can be quickly assembled from the human-readable text of JSON.
To the satisfaction of innovative database administration professionals, Teradata 15 is expected to possess greater application development features. Without having to attain new parallel programming skills, creators can now access a more diverse array of data and construct programs through a robust environment.
Organizing the disordered
One aspect of IoT data is that it is largely raw, unstructured and unorganized. As a result, IT-driven corporations are reassessing the value of NoSQL databases, which have been built to better handle the digital information produced by a wide array of smart devices, websites and other resources. According to Forbes, Oracle experts have claimed that this newfound interest presents a great opportunity to the world's second largest software company. The source noted IDC Research's prediction that 90 percent of all data produced this decade will be disorganized.
A traditional relational database management system is incapable of processing the heterogeneous, non-alphanumeric data that has grown quite prevalent of late. Forbes acknowledged the value of deploying a blog, which may possess carefully placed advertisements that drive the proprietor's revenue. Database analytics tools that skillfully select these instant marketing campaigns can only be supported by NoSQL, as it offers users horizontal scaling.
RDBMS is slowly fading out of the mixture, giving way to a new breed of operations better acclimated to the current disorganized data climate. In turn, database support services will answer with new assessment and management tools capable of handling NoSQL operations.
Businesses are realizing that investment in data analytics tools can be a major boon to their market intelligence divisions. Digital information collected from smartphones, tablets and other devices is contributing to a seemingly limitless vat of knowledge for executives looking to launch the next big product or service. Due to the sheer scale and complexity of such an operation, corporations are outsourcing their IT responsibilities to database administration services capable of giving them a comprehensive view of all market and in-house insight.
Connecting to more, in unlikely places
Innovators have speculated that a smart refrigerator may emerge in the not-too-distant future, capable of providing owners with recipes obtained from the Internet based upon what food is being stored in the machine. It's this kind of intelligence that could potentially revolutionize the food industry. Grocery stores may build entire marketing campaigns based upon what edibles are most popular. Chain restaurants may use these devices to store their products and funnel the information into customized analytics tools designed to create new recipes.
Behind these developments is the Internet of Things. Lori MacVittie, a contributor to InformationWeek, claimed that the IoT will continue to be integrated with wearables, children's toys, pens and other items, a process that's sure to revolutionize the contemporary data center. However, she noted that harnessing these assets won't be an easy endeavor, even for the most capable enterprises. Ultimately, two optional procedures will most likely become necessities. First, database experts will need to be consulted. Second, the appropriate applications will need to be used in order to process the influx of information.
How the consumer will use it
It's generally understood how corporations will harness this technology, but many are still speculating on how consumers will interact with devices connected to IoT. According to Business Insider contributor Emily Adler, the instruments people use on a daily basis will enter the widespread data environment. She noted that home appliances may be consulted by homeowners to determine how much energy a household is spending, the residual effect being that an individual can determine how he or she can reduce their electric bill.
As fitness is an ongoing trend that is likely to remain prevalent over the next couple of decades, chain gyms and exercise centers could ask their customers to connect their wearable devices to their corporate databases. Already, these mechanisms are capable of recording how many steps a person has taken as well as their weight, blood pressure and other statistics.
This phenomenon will likely result in a trade-off between consumers and businesses, the latter consulting database administration support to harness programs capable of translating customer data into actionable intelligence.
Though it may sound counterintuitive, a number of database experts have claimed that a company may benefit from disclosing information regarding its IT infrastructure to competitors. This may seem like a network security nightmare in and of itself, but collaborating with other market participants may provide valuable insight as to how organizations can deter cybercriminals. Others prefer to stick with improvements issued by established professionals.
Possessing quality database protection is being seen more as a profit-driver than an expense, primarily due to the fact that if digital information is stolen from a corporate server, it could potentially result in millions of dollars in losses. It's no surprise that database administration services are being consulted now more than ever. In addition, the makers of the products these professionals interact with have assessed security concerns and sought to mitigate potential problems.
Oracle NoSQL Database 3.0 was recently released, with improved performance, usability and safeguards. The upgrade utilizes cluster-wide, password-based user authentication and session-level SSL encryption techniques to deter cybercriminals from hacking into a company infrastructure. Andrew Mendelsohn, executive vice president of database server technologies for Oracle, claimed that that it helps remote DBA personnel construct and deploy state-of-the-art applications in a secure environment.
Walking around naked
Corporations often misunderstand the advice of IT professionals to share security protocols with their competitors. It's not about exposing weaknesses to cybercriminals and providing them with a comprehensive framework of the database's infrastructure, it's about collaborating with like-minded executives attempting to find a solution to an issue that isn't going to disappear.
Evan Schuman, a contributor to Computerworld, cited Full Disclosure, an online community through which database administration support, C-suite personnel and IT professionals could publicly report network breaches and discuss methods through which security problems could be resolved.
Due to the fact that gray hat hackers could access the forum, researchers would notify software companies at least 30 days prior to posting on the website so that the developers could apply the appropriate patches beforehand. This kind of initiative identified problems before cybercriminals could exploit them. Unfortunately, to the dismay of its participants, rumors have been circulating that Full Disclosure will shut down in the near future.
"By not having this place to expose them, the vulnerabilities will remain hidden longer, they will remain unpatched longer, yet the attacks will keep coming," said an anonymous security manager for a retailer.
Ultimately, black hat hackers have extensive communities through which they can share the same kind of information professionals posting to Full Disclosure are. If the website goes dark, cybercriminals will still have networks of communication, while law-abiding IT industry participants will not.
In light of a study recently released by the Intergovernmental Panel on Climate Change, the database administration needs of public agencies and organizations are expected to expand significantly. As it was industrialization and innovation that incited this worldwide issue, the Internet of Things will continue to be used to identify the detrimental effects climate change has on particular ecosystems and economies of the world.
Patrick Thibodeau, a contributor to Computerworld, claimed that the IPCC's study acknowledged the importance of sensor networks to monitor the shifting global environment. Potentially, these devices could help government officials anticipate droughts, floods and natural disasters caused by rising temperatures. In addition, it is hoped that the mechanisms will identify ways to preserve water and food supplies as well as determine methods for reducing energy consumption.
If public authorities choose to acknowledge the recommendations of the IPCC, the influx of new data derived from the IoT is sure to increase network traffic, requiring the expertise of remote database support to ensure that all analytics programs are running efficiently. As it's somewhat ambiguous as to how these sensors will be deployed, the kinds of avenues through which information flows into networks may pose as a challenge to in-house IT departments.
An example of a new innovation
The types of devices the government and non-profit environmental agencies use are quite variable. Some may track the shifting tides across the Atlantic and the Pacific while others will determine the acidity of farming soil. If all the data collected by these devices is assembled onto a single server, outsourced database experts may be consulted to mitigate it all. It looks as if scientists have already taken the first step.
According to Space Daily, engineers from Europe developed the Sentinel-5 instrument, a mechanism which allows the continent's Copernicus program to monitor air quality around the globe. The article noted that atmospheric pollution is linked to millions of deaths around the world.
"The readings will help to both monitor and differentiate between natural and human-produced emissions, providing new insight on the human impact on climate," noted the news source.
Amassing and translating such an incredible amount of data will most likely necessitate the expertise of remote DBA to ensure that networks don't crash or overload. It's hoped that Copernicus, the world's first operational environmental surveillance system, will provide scientists with specific insight on how the earth's population can reduce emissions.
My previous blog post was about the SSIS Lookup task and how it really works. Now that I have shown that the Lookup task shouldn’t be used for one-to-many or many-to-many joins, let’s take a look at the Merge Join transformation task. If you follow along with this blog, you will learn a little tip that will eliminate the requirement for you to add a SORT transformation task within your data flow task.
Previously, we isolated our results sets down to one employee in the AdventureWorks database and joined two tables together. I’m going to do the same thing here. This time, I am going to introduce a third table into the join logic. The three tables are listed below:
- Employee Department History
Here is what it would look like via SSMS with T-SQL:
Let’s see how we can mimic this result set in SSIS without using T-SQL to join the three tables. First, I want to say it is not always going to be the best option not to use T-SQL instead of individual tasks in SSIS. I have learned over time that it is easier to write the join logic directly in you data flow source task sometimes. However, this is for demonstration purposes.
Let’s say you received a request to extract a result set, order the results set, and load it to another location. Here is what your package would look like in SSIS using the Merge Join transformation task:
Here are our results:
Notice, I used the SORT transformation task in the example above. I used this to depict what has to occur in a step by step approach:
- Extracted data from the Person and Employee Department History tables
- Sorted each result set
- Merged the two results into one using inner join logic
- Extracted data from the Departement table
- Sorted the first Joined result set and the Department result set
- Merge the Joined result set from Persons and Employee History with the Department table
Let’s talk about best practice for this example. This is where the Sort tip is introduced. Since we need an ordered result set per the request, we are using the merge transformation instead of the Union All task. Additionally, we used the Sort task. The Sort task can heavily impact the performance of an SSIS package, particularly when you have larger result sets than what we are going to extract from the AdventureWorks database.
Best practice is to bring in an ordered result set at the source and then merge your record sets. Well, how do you do that? Let’s walk through ordering your result set at the source and configuring your source to define the sorted column for merging your record sets.
First, we open the task and add our ORDER BY clause to our source.
Next, close the source task, right click on the same source task, and choose the Show Advanced Editor option.
There are two specifications in the Advanced Editor that need to be defined in order to make this work:
- Click on the Input and Output Properties tab
- Click on the OLE DB Source Outputs
- Change the IsSorted parameter to “True”
- Drill down into the OLE DB Source Output to Output columns.
- Click on your columns that you used in your ORDER BY clause.
- Change your SortKeyPosition parameter from “0″ to “1″.
The desired results should look similar to those below:
Next, you can remove each sort task that directly follows your OLE DB Source task by repeating the steps above to reconfigure each source editor. Now, my data flow task looks like this:
We get back the same results:
In case you are wondering why I got rid of all of the Sort tasks except for the one that follows the first merge join, I’ll explain. There are two reasons for this. My second join is on DepartmentID and, most importantly, the merge transformation task is not considered a data flow source task and does not come with the functionality to define the sorted order.
To conclude my second blog post of this series, the Merge Join transformation task can be used to merge columns from two different tables using Join logic similar to the Joins that can be used in T-SQL. We have looked at a step by step break down of what has to occur to implement a Merge Join transformation task as well as discussing some tips and best practice in regards to using the Sort task in SSIS.
I hope this blog post has been informative and that you look forward to reading my third post soon.
Retailers that have failed to adapt to the e-commerce landscape are seemingly destined for failure. Those that have executed an omnichannel product delivery approach have implemented complex data analytics programs to provide them with valuable market insight on both individual customers and groups of people. Helping them effectively manage this software are database experts well acquainted with the technology.
Although online shopping has driven profits for merchandisers, it's also presented them with a list of new problems. One challenge that has evolved with the prevalence of e-commerce is reaching customer satisfaction. Back in the days when the only place to purchase items was in a brick-and-mortar store, it was enough to deliver a product that functioned the way it was supposed to at a reasonable price. Now, retail websites are expected to possess customized marketing campaigns for particular visitors and offer more rewards to loyal customers.
Meyar Sheik, CEO of personalization software vendor Certona, claimed that without the appropriate data and actions to target shoppers with relevant, applicable information, it becomes very difficult for merchandisers to execute an effective omnichannel strategy. In this respect, possessing the programs capable of managing and translating such a large amount of data is just as much a part of the the customer relations strategy as product development.
Leaving it to the professionals
As more retail executives are more concerned about the intelligence derived from the data, many have hired database administration services to effectively implement and run the data analytics programs. In a way, these professionals do more than maintain a corporate network, they provide the expertise and tools necessary to keep the foundation of a business profitable.
C-suite merchandisers aren't ignorant of this fact, either. According to a report released by research firm IDC, retailers are expected to heavily invest in big data and analytics projects in 2014, requiring the invaluable knowledge of IT services providers. In addition, the study showed that mobile applications connected to e-commerce operations will grow at a compound annual growth rate of 10 percent over the next three years.
From what can be gathered based on the latter statistic, smartphones and tablets are anticipated to be major participants in omnichannel strategies. It is very likely that database administration companies will be hired to oversee the connectivity of these devices and ensure that the avenues through which they communicate are not exploited by cybercriminals.
Overall, the functionality of data analytics tools and e-commerce software is dependent on IT professionals capable of assessing the needs of particular merchandisers.
Due to the limited capabilities of a 24-person IT department faced with data analytics programs, many organizations have turned to database administration experts to monitor and operate them. Though they may not deploy the systems themselves, an outsourced service well acquainted with specific client operations can provide valuable insight for business executives looking to gain actionable digital information.
An unlikely friend
Organizations providing data analytics systems often push their products as "one size fits all" programs that may or may not be applicable to businesses engaged in different industries. Database administration services acknowledge the specific needs of each of their clients and how they intend to use digital information processing software. Some may collect real-time data points on individual shopping habits while others may be using predictive tools to anticipate product backorders during an impending snow storm.
According to CIO Magazine, rental-car company Hertz supplements its in-house analytics resources and data center with an outsourced IT service provider. Barbara Wixom, a business intelligence expert at the Massachusetts Institute of Technology, claimed that the nationwide organization relies on the database experts to purge unnecessary information, host and manage data and provide insights. One of the programs the company utilizes examines comments from Web surveys, emails and text messages so that store managers can get a better view of customer satisfaction.
Connecting with the rest of the company
As database administrator services encounter hundreds, if not thousands of different data analytics programs in a typical work week, their personnel have obtained the invaluable ability to communicate the results of the programs to the people utilizing them. Predictive analytics tools provide actionable results, but learning how they work can be a daunting task for marketing professionals just trying to get market insight on particular individuals or populations.
Ron Bodkin, a contributor to InformationWeek, noted that acclimating individual departments to specific data processing actions is essential to the survival of a company. The writer cited Hitachi Global Storage Technologies, which created a data processing platform capable of hosting each team's separate needs and desires while still providing executives with a holistic view of all operations.
"Access to internal data often requires IT to move from limiting access for security to encouraging sharing while still governing access to data sets," claimed Bodkin.
The writer also acknowledged the importance of a general willingness to learn. Who better than database experts to educate unknowledgeable executives in how analytics programs operate?