Data Cleaning & Related Platform

-
1. Cascading (Apache) 
The Cascading Ecosystem is a collection of applications, languages, and APIs for developing data-intensive applications. 
At the ecosystem core is Cascading, a Java API for defining complex data flows and integrating those flows with back-end systems, and a query planner for mapping and executing logical flows onto a computing platform. 
There are quite a few extensions to Cascading providing integrations with popular systems, testing frameworks, and tools that leverage Cascading. 
Sitting on top of the Cascading API are languages and tools to simplify the development of data-intensive applications. For Scala developers, see Scalding. For Clojure developers, see Cascalog. For SQL developers, see Lingual. And for Java developers, the raw Cascading API can be used, or a fluent interface named Fluid. 
Sitting below the Cascading query planner are platform providers and rules for mapping data flows onto a given platform like Apache Hadoop, Apache Tez, Apache Flink, or simply locally in memory (suitable for many streaming applications). 
Learn more from the the User Guide, the most recent Cascading and Scalding books, or the tutorials and example applications. To learn about Cascading internals, see this post on the 3.x query planner. 

2. Talend 
Talend Open Studio for Data Integration helps you get your data to the right place, in the right form, at the right time. The leading open source ETL solution for data warehousing and business intelligence, Talend Open Studio for Data Integration is: 
  • Powerful and versatile. Transform, move, and synchronize data across all your heterogeneous sources and targets 
  • Easy to use. Start productive work right away with an intuitive interface rich in modelling tools, job-building components, and more than 900 data connectors, including the Cloud 
  • Proven in the field. Hundreds of thousands of users manage their critical data with Talend Open Studio for Data Integration, from SMBs to some of the largest corporations in the world 
  • Ready to start today. Talend Open Studio for Data Integration is free to download and use, for as long as you want. No budget battles or endless delays – just faster, easier data integration, starting today 

3. Pentaho 
business intelligence (BI) software that provides data integration, OLAP services, reporting, information dashboards, data mining and extract, transform, load (ETL) capabilities. It is headquartered in Orlando, Florida. Pentaho was acquired by Hitachi Data Systems in 2015. On September 19, 2017, Pentaho became part of Hitachi Vantara, a new company that unifies the operations of Pentaho, Hitachi Data Systems and Hitachi Insight Group. 
  • Internet of Things Analytics. Expect better business outcomes – from improved customer satisfaction to higher profitability –with the power of IoT analytics. 
  • Big Data Integration and Analytics. Drive maximum value from your data with a complete platform for full data integration and business analytics. 
  • Pentaho Data Integration. Quickly and easily deliver the best data to your business and IT users – no coding or complexity required. 
  • Business Analytics. Empower business users with interactive, real-time visual data analysis and predictive modelling, with minimal IT support. 

4. Sqoop (Apache) 
Apache Sqoop is a tool designed for efficiently transferring bulk data between Apache Hadoop and structured datastores such as relational databases. You can use Sqoop to import data from external structured datastores into Hadoop Distributed File System or related systems like Hive and HBase. Conversely, Sqoop can be used to extract data from Hadoop and export it to external structured datastores such as relational databases and enterprise data warehouses. 

Popular posts from this blog

Kokology Questions & Answers

-
[These are extracts only, to see the full details kindly purchase the original book] Kokology is the study of kokoro (Japanese: 心) which in the aforementioned language means "mind" or "spirit". ... The books were published in 1998 in Japan and became a Japanese bestselling phenomenon. The books were translated and became available in the United States in 2000. Below are some questions and answers for sharing: A Desert Journey Boring class settings, you drifted away and enter the eternal desert... 1. You are riding a camel and you are near exhaustion, what will you say to the camel? 2. Before you die of thirst, a traveler point to you a oasis, who is this person you know? 3. You finally come to your destination which is a town, what is your feeling? 4. After you dismount, another person took over, who is this guy? - 1. What happened when love is lost? 2. Who always help or comforted you? 3. What is your feeling after getting past lost love?...
Read more

Psychological Terms, Physics Laws & Effect, Mathematics & Paradoxes, Fallacies, Metaheuristics(Growing List)

-
Image
Psychology Analysis Paralysis It is the state of over-analyzing (or over-thinking) a situation so that a decision or action is never taken, in effect paralysing the outcome. A decision can be treated as over-complicated, with too many detailed options, so that a choice is never made, rather than try something and change if a major problem arises. A person might be seeking the optimal or "perfect" solution upfront, and fear making any decision which could lead to erroneous results, while on the way to a better solution. Antifragility A property of systems that increase in capability to thrive as a result of stressors, shocks, volatility, noise, mistakes, faults, attacks, or failures. Barnum effect Also called the Forer effect, or less commonly, the Barnum-Forer effect, is a common psychological phenomenon whereby individuals give high accuracy ratings to descriptions of their personality that supposedly are tailored specifically to them, that are in fact vague and general en...
Read more

The Art of Thinking Clearly (Rolf Dobelli, 2013)

-
Image
Survivorship bias means this: people systematically overestimate their chances of success. Guard against it by frequently visiting the graves of once-promising projects, investments and careers. It is a sad walk, but one that should clear your mind. Be wary when you are encouraged to strive for certain things – be it abs of steel, immaculate looks, a higher income, a long life, a particular demeanour or happiness. You might fall prey to the swimmer’s body illusion. Before you decide to take the plunge, look in the mirror – and be honest about what you see. When it comes to pattern recognition, we are oversensitive. Regain your scepticism. If you think you have discovered a pattern, first consider it pure chance. If it seems too good to be true, find a mathematician and have the data tested statistically. And if the crispy parts of your pancake start to look a lot like Jesus’ face, ask yourself: if he really wants to reveal himself, why doesn’t he do it in Times Square or on CNN? Be sce...
Read more