"With the advent of the Web along with the unprecedented amount of information available in electronic format, conceptual data analysis is more useful and practical than ever, because this technology addresses important limitations of the systems that currently support users in their quest for information.Показать полностью"With the advent of the Web along with the unprecedented amount of information available in electronic format, conceptual data analysis is more useful and practical than ever, because this technology addresses important limitations of the systems that currently support users in their quest for information. Concept Data Analysis: Theory & Applications is the first book that provides a comprehensive treatment of the full range of algorithms available for conceptual data analysis, spanning creation, maintenance, display and manipulation of concept lattices. The accompanying website allows you to gain a greater understanding of the principles covered in the book through actively working on the topics discussed. The three main areas explored are interactive mining of documents or collections of documents (including Web documents), automatic text ranking, and rule mining from structured data. The potentials of conceptual data analysis in the application areas being considered are further illustrated by two detailed case studies.

In this book, outspoken database magazine columnist Joe Celko waxes philosophic about fundamental concepts in database design and development. He points out misconceptions and plain ol' mistakes commonly made while creating databases including mathematical calculation errors, inappropriate key field choices, date representation goofs and more.Показать полностьюIn this book, outspoken database magazine columnist Joe Celko waxes philosophic about fundamental concepts in database design and development. He points out misconceptions and plain ol' mistakes commonly made while creating databases including mathematical calculation errors, inappropriate key field choices, date representation goofs and more. Celko also points out the quirks in SQL itself. A detailed table-of-contents will quickly route you to your area of interest.

Mining the Web: Discovering Knowledge from Hypertext Data is the first book devoted entirely to techniques for extracting and producing knowledge from the vast body of unstructured Web data.Показать полностьюMining the Web: Discovering Knowledge from Hypertext Data is the first book devoted entirely to techniques for extracting and producing knowledge from the vast body of unstructured Web data. Building on an initial survey of infrastructural issues—including Web crawling and indexing—Chakrabarti examines machine learning techniques as they relate specifically to the challenges of Web mining and provides applications of machine learning to sytematically acquire, store, and analyze data. Here the focus is on results: the strengths and weaknesses of these applications, along with their potential as foundations for further progress toward a Web that is more aware of content semantics. This thorough and forward-looking book gives the theoretical and practical foundations you need to build innovative applications for mining the Web. Features * A comprehensive, critical exploration of statistics-based attempts to make sense of Web data. * Details the special challenges associated with analyzing unstructured and semi-structured data. * Looks at how classical Information Retrieval techniques have been modified for use with Web data. * Focuses on today’s dominant learning methods: clustering and classification, hyperlink analysis, and supervised and semi-supervised learning. * Analyzes current applications for resource discovery and social network analysis. * An excellent way to introduce students to especially vital applications of data mining and machine learning technology.

* Written for practitioners of data mining, data cleaning and database management. * Presents a technical treatment of data quality including process, metrics, tools and algorithms. * Focuses on developing an evolving modeling strategy through an iterative data exploration loop and incorporation of domain knowledge. * Addresses methods of detecting, quantifying and correcting data quality issues that can have a significant impact on findings and decisions, using commercially available tools as well as new algorithmic approaches. * Uses case studies to illustrate applications in real life scenarios. * Highlights new approaches and methodologies, such as the DataSphere space partitioning and summary based analysis techniques.Показать полностью* Written for practitioners of data mining, data cleaning and database management. * Presents a technical treatment of data quality including process, metrics, tools and algorithms. * Focuses on developing an evolving modeling strategy through an iterative data exploration loop and incorporation of domain knowledge. * Addresses methods of detecting, quantifying and correcting data quality issues that can have a significant impact on findings and decisions, using commercially available tools as well as new algorithmic approaches. * Uses case studies to illustrate applications in real life scenarios. * Highlights new approaches and methodologies, such as the DataSphere space partitioning and summary based analysis techniques. Exploratory Data Mining and Data Cleaning will serve as an important reference for serious data analysts who need to analyze large amounts of unfamiliar data, managers of operations databases, and students in undergraduate or graduate level courses dealing with large scale data analys is and data mining.

The field of Socially Intelligent Agents (SIA) is a fast growing and increasingly important area that comprises highly active research activities and strongly interdisciplinary approaches.Показать полностьюThe field of Socially Intelligent Agents (SIA) is a fast growing and increasingly important area that comprises highly active research activities and strongly interdisciplinary approaches. Socially Intelligent Agents, edited by Kerstin Dautenhahn, Alan Bond, Lola Canamero and Bruce Edmonds, emerged from the AAAI Symposium "Socially Intelligent Agents — The Human in the Loop". The book provides 32 chapters, written by leading SIA researchers, addressing topics such as: social robotics, embodied conversational agents, affective computing, anthropomorphism, narrative and story-telling, social aspects in multi-agent systems, new technologies for education and therapy, and more. This breadth of topics covered in Socially Intelligent Agents provides the reader with a comprehensive look at current research activities in the area. Socially Intelligent Agents serves as an excellent reference for a wide readership, e.g. computer scientists, roboticists, web programmers and designers, computer users, cognitive scientists, and other researchers interested in the study of how humans relate to computers and robots, and how these agents in return can relate to humans. This book is also suitable as research material in a variety of advanced level courses, including Applied Artificial Intelligence, Autonomous Agents, Human-Computer Interaction, Situated, Embodied AI.

Marketing analysts use data mining techniques to gain a reliable understanding of customer buying habits and then use that information to develop new marketing campaigns and products.Показать полностьюMarketing analysts use data mining techniques to gain a reliable understanding of customer buying habits and then use that information to develop new marketing campaigns and products. Visual mining tools introduce a world of possibilities to a much broader and non-technical audience to help them solve common business problems. * Explains how to select the appropriate data sets for analysis, transform the data sets into usable formats, and verify that the sets are error-free * Reviews how to choose the right model for the specific type of analysis project, how to analyze the model, and present the results for decision making * Shows how to solve numerous business problems by applying various tools and techniques * Companion Web site offers links to data visualization and visual data mining tools, and real-world success stories using visual data mining

A readable introduction aimed at business users who want a clear, non-technical overview of the techniques and capabilities of data mining. A valuable educational tool for prospective users of this exciting new technology. Topics covered include Data Description for Data Mining, Predictive Data Mining, Data Mining Models and Algorithms, The Data Mining Process, and Selecting Data Mining Products.

A Bayesian network is a graphical model that encodes probabilistic relationships among variables of interest. When used in conjunction with statistical techniques, the graphical model has several advantages for data modeling.Показать полностьюA Bayesian network is a graphical model that encodes probabilistic relationships among variables of interest. When used in conjunction with statistical techniques, the graphical model has several advantages for data modeling. One, because the model encodes dependencies among all variables, it readily handles situations where some data entries are missing. Two, a Bayesian network can be used to learn causal relationships, and hence can be used to gain understanding about a problem domain and to predict the consequences of intervention. Three, because the model has both a causal and probabilistic semantics, it is an ideal representation for combining prior knowledge (which often comes in causal form) and data. Four, Bayesian statistical methods in conjunction with Bayesian networks offer an efficient and principled approach for avoiding the overfitting of data. In this paper, we discuss methods for constructing Bayesian networks from prior knowledge and summarize Bayesian statistical methods for using data to improve these models. With regard to the latter task, we describe methods for learning both the parameters and structure of a Bayesian network, including techniques for learning with incomplete data. In addition, we relate Bayesian-network methods for learning to techniques for supervised and unsupervised learning. We illustrate the graphical-modeling approach using a real-world case study.

In an information intensive age, data is the fuel powering the organizational machine. Flowing through the hands of knowledge workers, it is used to determine everything from corporate strategy and direction to product development and advertising concepts.Показать полностьюIn an information intensive age, data is the fuel powering the organizational machine. Flowing through the hands of knowledge workers, it is used to determine everything from corporate strategy and direction to product development and advertising concepts. Companies must have easy and instant access to data if they want to maintain a competitive edge. Data Warehousing Advice for Managers explores the possibilities of the “data warehouse” - -a data storage technology that optimizes accessibility — and assists managers in determining whether it’s right for their organization. It helps readers: ** understand the benefits that data warehousing offers and convince upper management to take action ** coordinate the data warehouse with other technologies ** manage the implementation of the warehouse to ensure its compliance to specified requirements ** obtain the highest return on their investment.

We have been using fuzzy data mining techniques to extract patterns that represent normal behavior for intrusion detection. In this paper we describe a variety of modifications that we have made to the data mining algorithms in order to improve accuracy and efficiency.Показать полностьюWe have been using fuzzy data mining techniques to extract patterns that represent normal behavior for intrusion detection. In this paper we describe a variety of modifications that we have made to the data mining algorithms in order to improve accuracy and efficiency. We use sets of fuzzy association rules that are mined from network audit data as models of “normal behavior.” To detect anomalous behavior, we generate fuzzy association rules from new audit data and compute the similarity with sets mined from “normal” data. If the similarity values are below a threshold value, an alarm is issued. In this paper we describe an algorithm for computing fuzzy association rules based on Borgelt’s prefix trees, modifications to the computation of support and confidence of fuzzy rules, a new method for computing the similarity of two fuzzy rule sets, and feature selection and optimization with genetic algorithms. Experimental results demonstrate that we can achieve better running time and accuracy with these modifications.

This book offers a thorough grounding in machine learning concepts as well as practical advice on applying machine learning tools and techniques in real-world data mining situations.Показать полностьюThis book offers a thorough grounding in machine learning concepts as well as practical advice on applying machine learning tools and techniques in real-world data mining situations. Inside, you'll learn all you need to know about preparing inputs, interpreting outputs, evaluating results, and the algorithmic methods at the heart of successful data miningincluding both tried-and-true techniques of the past and Java-based methods at the leading edge of contemporary research. If you're involved at any level in the work of extracting usable knowledge from large collections of data, this clearly written and effectively illustrated book will prove an invaluable resource. Complementing the authors' instruction is a fully functional platform-independent Java software system for machine learning, available for download. Apply it to the sample data sets provided to refine your data mining skills, apply it to your own data to discern meaningful patterns and generate valuable insights, adapt it for your specialized data mining applications, or use it to develop your own machine learning schemes. * Helps you select appropriate approaches to particular problems and to compare and evaluate the results of different techniques. * Covers performance improvement techniques, including input preprocessing and combining output from different methods. * Comes with downloadable machine learning software: use it to master the techniques covered inside, apply it to your own projects, and/or customize it to meet special needs.

Here is an invaluable overview of all the cutting-edge features of Oracle’s latest database release, Oracle Database 10g. Includes expert commentary throughout from world-renowned Oracle guru Jonathan Lewis. This is an ideal resource for decision-makers and IT staff preparing for upgrades or migration.

This chapter discusses the use of evolutionary algorithms, particularly genetic algorithms and genetic programming, in data mining and knowledge discovery.Показать полностьюThis chapter discusses the use of evolutionary algorithms, particularly genetic algorithms and genetic programming, in data mining and knowledge discovery. We focus on the data mining task of classification. In addition, we discuss some preprocessing and postprocessing steps of the knowledge discovery process, focusing on attribute selection and pruning of an ensemble of classifiers. We show how the requirements of data mining and knowledge discovery influence the design of evolutionary algorithms. In particular, we discuss how individual representation, genetic operators and fitness functions have to be adapted for extracting high-level knowledge from data.

Data mining, or Knowledge Discovery in Databases (KDD), is of little benefit to commercial enterprises unless it can be carried out efficiently on realistic volumes of data.Показать полностьюData mining, or Knowledge Discovery in Databases (KDD), is of little benefit to commercial enterprises unless it can be carried out efficiently on realistic volumes of data. Operational factors also dictate that KDD should be performed within the context of standard DBMS. Fortunately, relational DBMS have a declarative query interface (SQL) that has allowed designers of parallel hardware to exploit data parallelism efficiently. Thus, an effective approach to the problem of efficient KDD consists of arranging that KDD tasks execute on a parallel SQL server. In this paper we devise generic KDD primitives, map these to SQL and present some results of running these primitives on a commercially-available parallel SQL server.

During the past decade there has been an explosion in computation and information technology. With it has come vast amounts of data in a variety of fields such as medicine, biology, finance, and marketing.Показать полностьюDuring the past decade there has been an explosion in computation and information technology. With it has come vast amounts of data in a variety of fields such as medicine, biology, finance, and marketing. The challenge of understanding these data has led to the development of new tools in the field of statistics, and spawned new areas such as data mining, machine learning, and bioinformatics.Many of these tools have common underpinnings but are often expressed with different terminology. This book describes the important ideas in these areas in a common conceptual framework. While the approach is statistical, the emphasis is on concepts rather than mathematics. Many examples are given, with a liberal use of color graphics. It should be a valuable resource for statisticians and anyone interested in data mining in science or industry. The book’s coverage is broad, from supervised learning (prediction) to unsupervised learning. The many topics include neural networks, support vector machines, classification trees and boosting—the first comprehensive treatment of this topic in any book. Trevor Hastie, Robert Tibshirani, and Jerome Friedman are professors of statistics at Stanford University. They are prominent researchers in this area: Hastie and Tibshirani developed generalized additive models and wrote a popular book of that title. Hastie wrote much of the statistical modeling software in S-PLUS and invented principal curves and surfaces. Tibshirani proposed the Lasso and is co-author of the very successful An Introduction to the Bootstrap. Friedman is the co-inventor of many data-mining tools including CART, MARS, and projection pursuit.

This book is an introduction to pattern recognition, meant for undergraduate and graduate students in computer science and related fields in science and technology.Показать полностьюThis book is an introduction to pattern recognition, meant for undergraduate and graduate students in computer science and related fields in science and technology. Most of the topics are accompanied by detailed algorithms and real world applications. In addition to statistical and structural approaches, novel topics such as fuzzy pattern recognition and pattern recognition via neural networks are also reviewed. Each topic is followed by several examples solved in detail. The only prerequisites for using this book are a one-semester course in discrete mathematics and a knowledge of the basic preliminaries of calculus, linear algebra and probability theory.

Finding information hidden in data is as theoretically difficult as it is practically important. With the objective of discovering unknown patterns from data, the methodologies of data mining were derived from statistics, machine learning, and artificial intelligence, and are being used successfully in application areas such as bioinformatics, banking, retail, and many others.Показать полностьюFinding information hidden in data is as theoretically difficult as it is practically important. With the objective of discovering unknown patterns from data, the methodologies of data mining were derived from statistics, machine learning, and artificial intelligence, and are being used successfully in application areas such as bioinformatics, banking, retail, and many others. Wang and Fu present in detail the state of the art on how to utilize fuzzy neural networks, multilayer perceptron neural networks, radial basis function neural networks, genetic algorithms, and support vector machines in such applications. They focus on three main data mining tasks: data dimensionality reduction, classification, and rule extraction. The book is targeted at researchers in both academia and industry, while graduate students and developers of data mining systems will also profit from the detailed algorithmic descriptions.

Summary information from data in large databases is used to answer queries in On-Line Analytical Processing (OLAP) systems and to build decision support systems over them.Показать полностьюSummary information from data in large databases is used to answer queries in On-Line Analytical Processing (OLAP) systems and to build decision support systems over them. The Data Cube is used to calculate and store summary information on a variety of dimensions, which is computed only partially if the number of dimensions is large. Queries posed on such systems are quite complex and require different views of data. These may either be answered from a materialized cube in the data cube or calculated on the fly. Further, data mining for associations can be performed on the data cube. Analytical models need to capture the multidimensionality of the underlying data, a task for which multidimensional databases are well suited. Also, they are amenable to parallelism, which is necessary to deal with large (and still growing) data sets. Multidimensional databases store data in multidimensional structure on which analytical operations are performed. A challenge for these systems is how to handle large data sets in a large number of dimensions. These techniques are also applicable to scientific and statistical databases (SSDB) which employ large multidimensional databases and dimensional operations over them. In this paper we present (1) A parallel infrastructure for OLAP multidimensional databases integrated with association rule mining. (2) Introduce Bit-Encoded Sparse Structure (BESS) for sparse data storage in chunks. (3) Scheduling optimizations for parallel computation of complete and partial data cubes. (4) Implementation of a large scale multidimensional database engine suitable for dimensional analysis used in OLAP and SSDB for (a) large number of dimensions (20-30) (b) large data sets (10s of Gigabyte) Our implementation on the IBM SP-2 can handle large data sets and a large number of dimensions by using disk I/O. Results are presented showing its performance and scalability.

Information Retrieval (IR) is the discipline that deals with retrieval of unstructured data, especially textual documents, in response to a query or topic statement, which may itself be unstructured, e.g., a sentence or even another document, or which may be structured, e.g., a boolean expression.Показать полностьюInformation Retrieval (IR) is the discipline that deals with retrieval of unstructured data, especially textual documents, in response to a query or topic statement, which may itself be unstructured, e.g., a sentence or even another document, or which may be structured, e.g., a boolean expression. The need for effective methods of automated IR has grown in importance because of the tremendous explosion in the amount of unstructured data, both internal, corporate document collections, and the immense and growing number of document sources on the Internet. This report is a tutorial and survey of the state of the art, both research and commercial, in this dynamic field. The topics covered include: formulation of structured and unstructured queries and topic statements, indexing (including term weighting) of document collections, methods for computing the similarity of queries and documents, classification and routing of documents in an incoming stream to users on the basis of topic or need statements, clustering of document collections on the basis of language or topic, and statistical, probabilistic, and semantic methods of analyzing and retrieving documents.

This course focuses on software agents, particularly mobile agents. The programming language used is Java. Several agent API’s are discussed. These include Aglets, originally from IBM, now Open Source, the Java Agent Development Environment (JADE) from the University of Parma, and Ascape, from the Brookings Institute in Washington DC.Показать полностьюThis course focuses on software agents, particularly mobile agents. The programming language used is Java. Several agent API’s are discussed. These include Aglets, originally from IBM, now Open Source, the Java Agent Development Environment (JADE) from the University of Parma, and Ascape, from the Brookings Institute in Washington DC. Communication languages such as the Semantic Language (SL) and XML will also be discussed.