Data Mining for Connecting the Dots

Ted Senator, DARPA

Data mining has been proposed as a technique to "connect the dots" - i.e., to find interesting patterns in, from, and of linked data. However, traditional data mining processes and techniques are not well suited to these types of applications.  Using examples from several domains, this talk discusses the distinct requirements, technical challenges, and research issues of analyzing relational data to find instances of complex structured patterns in linked data.  It covers methods for data preparation; processes and techniques for pattern discovery, evaluation, and management; and detection strategies.  It also presents mathematical models of the problems and potential solutions.


Ted Senator is a Program Manager in the Information Processing Technology Office (IPTO) of the Defense Advanced Research Projects Agency (DARPA).  He was most recently responsible for Evidence Extraction and Link Discovery (EELD), a program begun before 9/11 designed specifically to develop technology for "connecting the dots" and for Bio-ALIRT, a program that investigates the utility of various non-traditional data sources for early detection of a potential biological outbreak.  He has 14 years experience in building applications to detect potential improper activity from large amounts of data.  Prior to coming to DARPA, he founded and headed the knowledge discovery and data mining group at NASD Regulation, leading the team that developed the system used to monitor the Nasdaq stock market for improper trading and quotation activity.  He was chief of the Artificial Intelligence Division of the Financial Crimes Enforcement Network (FinCEN) of the US Treasury Department, where he designed and implemented the system that detects potential money laundering from reports of large cash transactions.  He has degrees in Physics and in Electrical Engineering from MIT, and has done additional graduate study in physics, computer science, AI, and finance.