Just Scraping Alpha
This commit is contained in:
Rumps
parent
75e3dc5790
commit
b6c1660f04
24 changed files with 8 additions and 1000 deletions
|
|
@ -1,16 +1,16 @@
|
||||||
# Scraping Alpha
|
# Scraping Alpha
|
||||||
|
|
||||||
### Author
|
## Author
|
||||||
|
|
||||||
Ben Goldsworthy
|
Ben Goldsworthy
|
||||||
<[email](mailto:b.goldsworthy@lancaster.ac.uk)>
|
<[email](mailto:b.goldsworthy@lancaster.ac.uk)>
|
||||||
<[website](http://www.bengoldsworthy.uk/)>
|
<[website](http://www.bengoldsworthy.uk/)>
|
||||||
|
|
||||||
### Version
|
## Version
|
||||||
|
|
||||||
1.0
|
1.0
|
||||||
|
|
||||||
### Abstract
|
## Abstract
|
||||||
|
|
||||||
Scraping Alpha is a series of Python scripts used to scrape
|
Scraping Alpha is a series of Python scripts used to scrape
|
||||||
[Seeking Alpha](http://seekingalpha.com/) earnings call transcripts and produce
|
[Seeking Alpha](http://seekingalpha.com/) earnings call transcripts and produce
|
||||||
|
|
@ -18,13 +18,13 @@ SQL from them.
|
||||||
|
|
||||||
It was created for Dr Lars Hass of the Lancaster University Management School.
|
It was created for Dr Lars Hass of the Lancaster University Management School.
|
||||||
|
|
||||||
### Usage
|
## Usage
|
||||||
|
|
||||||
The instructions for each step of the process can be found at the beginning of
|
The instructions for each step of the process can be found at the beginning of
|
||||||
each of the files involved: `transcript_spider.py`, `JSONtoSQL.py` and
|
each of the files involved: `transcript_spider.py`, `JSONtoSQL.py` and
|
||||||
`execsAndAnalysts.py`. The are repeated here for brevity.
|
`execsAndAnalysts.py`. The are repeated here for brevity.
|
||||||
|
|
||||||
#### `transcript_spider.py`
|
### `transcript_spider.py`
|
||||||
|
|
||||||
This file is the webspider that Scrapy uses to retrieve the information from the
|
This file is the webspider that Scrapy uses to retrieve the information from the
|
||||||
website. Left unattended, it will scrape all 4,000+ pages of results.
|
website. Left unattended, it will scrape all 4,000+ pages of results.
|
||||||
|
|
@ -54,7 +54,7 @@ For installation instructions for Scrapy, see
|
||||||
transcripts -o transcripts.json` at the command line (the output file will be
|
transcripts -o transcripts.json` at the command line (the output file will be
|
||||||
placed in the directory the Terminal is currently pointing to).
|
placed in the directory the Terminal is currently pointing to).
|
||||||
|
|
||||||
#### `JSONtoSQL.py`
|
### `JSONtoSQL.py`
|
||||||
|
|
||||||
This file takes the `transcripts.json` file output of `transcript_spider.py` and
|
This file takes the `transcripts.json` file output of `transcript_spider.py` and
|
||||||
converts it into SQL.
|
converts it into SQL.
|
||||||
|
|
@ -63,7 +63,7 @@ This file should be located in the same directory as `transcripts.json`, and is
|
||||||
run via `python JSONtoSQL.py > [FILE].sql`, where `[FILE]` is the desired name
|
run via `python JSONtoSQL.py > [FILE].sql`, where `[FILE]` is the desired name
|
||||||
of the output file.
|
of the output file.
|
||||||
|
|
||||||
#### `execsAndAnalysts.py`
|
### `execsAndAnalysts.py`
|
||||||
|
|
||||||
First, import the output file of `JSONtoSQL.py` to your chosen DBMS (I've tested
|
First, import the output file of `JSONtoSQL.py` to your chosen DBMS (I've tested
|
||||||
it with phpMyAdmin). Then, run the following query:
|
it with phpMyAdmin). Then, run the following query:
|
||||||
|
|
@ -80,6 +80,6 @@ into your DBMS to create two linking tables. The final instruction of
|
||||||
`analysts.sql` then deletes the superfluous `execs` and `analysts` columns from
|
`analysts.sql` then deletes the superfluous `execs` and `analysts` columns from
|
||||||
the `transcripts` table (and for this reason, `execs.sql` must be imported first).
|
the `transcripts` table (and for this reason, `execs.sql` must be imported first).
|
||||||
|
|
||||||
### Future
|
## Future
|
||||||
|
|
||||||
Harvesting the URLs of slide images shouldn't be too hard to implement - `slides_spider.py` should in theory to this, but the link to a transcript's slides is added to the page later via Javascript, which means at the moment it throws up a load of HTTP 200 status codes and nowt else. [Scrapy+Splash](https://github.com/scrapy-plugins/scrapy-splash) may be the solution, however.
|
Harvesting the URLs of slide images shouldn't be too hard to implement - `slides_spider.py` should in theory to this, but the link to a transcript's slides is added to the page later via Javascript, which means at the moment it throws up a load of HTTP 200 status codes and nowt else. [Scrapy+Splash](https://github.com/scrapy-plugins/scrapy-splash) may be the solution, however.
|
||||||
|
|
@ -1,27 +0,0 @@
|
||||||
# PatentSlurp
|
|
||||||
Patent slurper for Dr Lars Hass, LUMS
|
|
||||||
|
|
||||||
# TODO
|
|
||||||
|
|
||||||
|
|
||||||
1) Add stripper for redundant xml tags
|
|
||||||
2) Harvest below data from Google dumps 2001-2015:
|
|
||||||
-------------------------------------------------------------------------------
|
|
||||||
storage display value
|
|
||||||
variable name type format label variable label
|
|
||||||
-------------------------------------------------------------------------------
|
|
||||||
sta str2 %2s assg/state
|
|
||||||
cnt str3 %3s assg/country
|
|
||||||
assgnum byte %8.0g assg/assignee seq. number (imc)
|
|
||||||
cty str72 %72s assg/city
|
|
||||||
pdpass long %12.0g Unique assignee number
|
|
||||||
ptype str1 %9s patent type
|
|
||||||
patnum long %12.0g patent number
|
|
||||||
-------------------------------------------------------------------------------
|
|
||||||
3) Compare data with NBER data (http://eml.berkeley.edu/~bhhall/NBER06.html)
|
|
||||||
4) ...
|
|
||||||
|
|
||||||
# Useful Tools
|
|
||||||
|
|
||||||
http://codebeautify.org/xmlviewer
|
|
||||||
http://www.regexr.com/
|
|
||||||
|
|
@ -1,183 +0,0 @@
|
||||||
#!/usr/bin/env perl
|
|
||||||
|
|
||||||
use 5.010;
|
|
||||||
#use strict;
|
|
||||||
use warnings;
|
|
||||||
use XML::Simple;
|
|
||||||
use XML::Twig;
|
|
||||||
use Time::Piece;
|
|
||||||
|
|
||||||
=pod
|
|
||||||
|
|
||||||
=head1 NAME
|
|
||||||
|
|
||||||
PATENT_SLURPER - A program to process Google/USPTO data dumps
|
|
||||||
|
|
||||||
=head1 VERSION
|
|
||||||
|
|
||||||
0.8
|
|
||||||
|
|
||||||
=head1 DESCRIPTION
|
|
||||||
|
|
||||||
This program takes L<Google/USPTO data dumps|https://www.google.com/googlebooks/uspto-patents-grants-text.html>
|
|
||||||
and produces SQL commands to generate a database from them with a
|
|
||||||
number of selected fields.
|
|
||||||
|
|
||||||
=head1 AUTHOR
|
|
||||||
|
|
||||||
L<Ben Goldsworthy (rumperuu)|mailto:b.goldsworthy96@gmail.com>
|
|
||||||
|
|
||||||
=head1 LICENSE
|
|
||||||
|
|
||||||
=cut
|
|
||||||
|
|
||||||
# Trims the file extension from the filename argument
|
|
||||||
my $filename = $ARGV[0];
|
|
||||||
chomp $filename;
|
|
||||||
|
|
||||||
my $patentTwig = new XML::Twig(TwigRoots => {
|
|
||||||
'SDOBI/B100/B140/DATE/PDAT' => 1,
|
|
||||||
'SDOBI/B100/B110/DNUM/PDAT' => 2,
|
|
||||||
'SDOBI/B200/B220/DATE/PDAT' => 3,
|
|
||||||
'SDOBI/B200/B210/DNUM/PDAT' => 4
|
|
||||||
},
|
|
||||||
TwigHandlers => {
|
|
||||||
'SDOBI/B100/B140/DATE/PDAT' => sub { $_->set_tag( 'appdate') },
|
|
||||||
'SDOBI/B100/B110/DNUM/PDAT' => sub { $_->set_tag( 'appnum') },
|
|
||||||
'SDOBI/B200/B220/DATE/PDAT' => sub { $_->set_tag( 'pubdate') },
|
|
||||||
'SDOBI/B200/B210/DNUM/PDAT' => sub { $_->set_tag( 'pubnum') }
|
|
||||||
},
|
|
||||||
pretty_print => 'indented');
|
|
||||||
my $citationTwig = new XML::Twig(TwigRoots => {
|
|
||||||
'SDOBI/B200/B210/DNUM/PDAT' => 1,
|
|
||||||
'SDOBI/B500/B560/B561/PCIT/DOC/DNUM/PDAT' => 2
|
|
||||||
},
|
|
||||||
TwigHandlers => {
|
|
||||||
'SDOBI/B200/B210/DNUM/PDAT' => sub { $_->set_tag( 'pubnum') },
|
|
||||||
'SDOBI/B500/B560/B561/PCIT/DOC/DNUM/PDAT' => sub { $_->set_tag('citing') }
|
|
||||||
},
|
|
||||||
pretty_print => 'indented');
|
|
||||||
|
|
||||||
my $numLines = countFile($filename);
|
|
||||||
print "Processing $numLines lines...\n";
|
|
||||||
processFile($filename, $numLines);
|
|
||||||
print "File processing finished - press '1' to generate SQL statements, or '0' to quit.\n";
|
|
||||||
while (1) {
|
|
||||||
given (<STDIN>) {
|
|
||||||
when(1) {
|
|
||||||
generateSQL($filename);
|
|
||||||
print "SQL generation finished.\n";
|
|
||||||
exit 0;
|
|
||||||
} when(0) {
|
|
||||||
exit 0;
|
|
||||||
} default {
|
|
||||||
print "Press '1' to generate SQL statements, or '0' to quit.\n";
|
|
||||||
}
|
|
||||||
};
|
|
||||||
}
|
|
||||||
|
|
||||||
# Goes through the file serially to count the lines
|
|
||||||
sub countFile {
|
|
||||||
my $lineNum = 0;
|
|
||||||
open(INFILE, "data/".$_[0].".xml") or die "Can't open ".$_[0].": $!";
|
|
||||||
foreach(<INFILE>) {
|
|
||||||
++$lineNum;
|
|
||||||
}
|
|
||||||
close(INFILE);
|
|
||||||
return $lineNum;
|
|
||||||
}
|
|
||||||
|
|
||||||
# Processes the file line-by-line, removing duplicate <?xml> and
|
|
||||||
# <!DOCTYPE> tags and extracting the fields listed above. It has to be
|
|
||||||
# done line-by-line (hence the use of XML::Twig rather than XML:Simple)
|
|
||||||
# rather than loading the entire .xml file into memory because the files
|
|
||||||
# are far too big to fit.
|
|
||||||
sub processFile {
|
|
||||||
my $buffer = "", my $firstItem = 1, my $currentLine = 1;
|
|
||||||
|
|
||||||
open(INFILE, "data/".$_[0].".xml") or die "Can't open ".$_[0].": $!";
|
|
||||||
unlink("details/".$_[0].".xml");
|
|
||||||
unlink("citations/".$_[0].".xml");
|
|
||||||
open(my $detailsFile, ">>details/".$_[0].".xml") or die "Can't output ".$_[0].": $!";
|
|
||||||
open(my $citationsFile, ">>citations/".$_[0].".xml") or die "Can't output ".$_[0].": $!";
|
|
||||||
|
|
||||||
# Prints the root node to the files for XML::Simple in generateSQL()
|
|
||||||
#print $detailsFile "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<PATDOC>";
|
|
||||||
#print $citationsFile "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<PATDOC>\n";
|
|
||||||
|
|
||||||
# For each line, build up a buffer (excluding the <?xml> and
|
|
||||||
# <!DOCTYPE> tags). When the next <us-patent-grant> item is reached
|
|
||||||
# (a.k.a. when the next <?xml> tag after the initial one is reached),
|
|
||||||
# run the buffer through the details and citations twigs and print
|
|
||||||
# the results to the relevant files. Then clear the buffer for the next
|
|
||||||
# <us-patent-grant>
|
|
||||||
foreach(<INFILE>) {
|
|
||||||
print "Processing line ".($currentLine++)."/".$_[1]."...\n";
|
|
||||||
|
|
||||||
#if ($_ !~ /^\<\?xml/ && $firstItem == 0) {
|
|
||||||
#if ($_ !~ /^\<\!DOCTYPE/) {
|
|
||||||
#}
|
|
||||||
#} elsif ($firstItem == 1) {
|
|
||||||
# $firstItem = 0;
|
|
||||||
#} else {
|
|
||||||
if ($_ =~ /^\<\?xml/ && $firstItem == 0) {
|
|
||||||
#print "\n----\n".$buffer."\n----\n";
|
|
||||||
$patentTwig->parse($buffer);
|
|
||||||
$patentTwig->print($detailsFile);
|
|
||||||
$citationTwig->parse($buffer);
|
|
||||||
$citationTwig->print($citationsFile);
|
|
||||||
|
|
||||||
$buffer = "";
|
|
||||||
} elsif ($_ !~ /^\<\!/ && $_ !~ /^\]\>/) {
|
|
||||||
if ($firstItem == 0) {
|
|
||||||
$string = $_;
|
|
||||||
$string =~ s/\&[a-zA-Z0-9]*;/zonk/g;
|
|
||||||
$buffer = $buffer.$string;
|
|
||||||
|
|
||||||
} else {
|
|
||||||
|
|
||||||
print $detailsFile "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<patents>";
|
|
||||||
print $citationsFile "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<patents>\n";
|
|
||||||
$firstItem = 0;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
print $detailsFile "</patents>";
|
|
||||||
print $citationsFile "</patents>";
|
|
||||||
|
|
||||||
close($detailsFile);
|
|
||||||
close($citationsFile);
|
|
||||||
close(INFILE);
|
|
||||||
}
|
|
||||||
|
|
||||||
# Generates an SQL dump of the database formed from analysing the .xml
|
|
||||||
# files.
|
|
||||||
sub generateSQL {
|
|
||||||
my $xml = new XML::Simple (KeyAttr=>[]);
|
|
||||||
|
|
||||||
my $details = $xml->XMLin("details/".$_[0].".xml");
|
|
||||||
|
|
||||||
unlink("sql/".$_[0].".sql");
|
|
||||||
open(my $sqlFile, ">>sql/".$_[0].".sql") or die "Can't output SQL ".$_[0].": $!";
|
|
||||||
|
|
||||||
print $sqlFile "CREATE TABLE IF NOT EXISTS `patent`\n(\n\t`pid` INT NOT NULL AUTO_INCREMENT,\n\t`pubNum` VARCHAR(32),\n\t`pubDate` DATETIME,\n\t`appNum` VARCHAR(32),\n\t`appDate` DATETIME,\n\tPRIMARY KEY(`pid`)\n);\n\nCREATE TABLE IF NOT EXISTS `patent_cite`\n(\n\t`citing_id` VARCHAR(32) NOT NULL,\n\t`cited_id` VARCHAR(32) NOT NULL,\n\tPRIMARY KEY (`citing_id`, `cited_id`)\n);\n\n";
|
|
||||||
print $sqlFile "INSERT INTO `patent` (`pubNum`, `pubDate`, `appNum`, `appDate`) VALUES";
|
|
||||||
foreach my $e (@{$details->{'PATDOC'}}) {
|
|
||||||
print $sqlFile "\n\t('".$e->{'pubnum'}."', '".$e->{'pubdate'}."', '".$e->{'appnum'}."', '".$e->{'appdate'}."'),";
|
|
||||||
}
|
|
||||||
print $sqlFile "\n\t('0', '0', '0', '0');\n\n-- This line and the above (0,0,0,0) tuple are needed due to the nature\n-- of the loop that builds the INSERT query, and the resultant SQL file\n-- being too long to edit from the end easily.\nDELETE FROM `patent` WHERE `pid` = '0';";
|
|
||||||
|
|
||||||
my $citations = $xml->XMLin("citations/".$_[0].".xml");
|
|
||||||
|
|
||||||
print $sqlFile "\n\nINSERT INTO `patent_cite` (`citing_id`, `cited_id`) VALUES";
|
|
||||||
foreach my $f (@{$citations->{'PATDOC'}}) {
|
|
||||||
my $pubNum = $f->{'pubnum'};
|
|
||||||
foreach (@{$f->{'citing'}}) {
|
|
||||||
print $sqlFile "\n\t('".$pubNum."', '".$_."'),";
|
|
||||||
}
|
|
||||||
}
|
|
||||||
print $sqlFile "\n\t('0', '0', '0', '0');\n\n-- This line and the above (0,0,0,0) tuple are needed due to the nature\n-- of the loop that builds the INSERT query, and the resultant SQL file\n-- being too long to edit from the end easily.\nDELETE FROM `patent_cite` WHERE `citing_id` = '0';";
|
|
||||||
|
|
||||||
close($sqlFile);
|
|
||||||
}
|
|
||||||
|
|
@ -1,401 +0,0 @@
|
||||||
INSERT INTO `analysts_to_transcripts` (`analyst_id`, `transcript_id`) VALUES
|
|
||||||
(1, 0),
|
|
||||||
(2, 1),
|
|
||||||
(3, 2),
|
|
||||||
(3, 3),
|
|
||||||
(3, 4),
|
|
||||||
(3, 5),
|
|
||||||
(3, 6),
|
|
||||||
(3, 7),
|
|
||||||
(3, 8),
|
|
||||||
(3, 9),
|
|
||||||
(3, 10),
|
|
||||||
(4, 11),
|
|
||||||
(4, 12),
|
|
||||||
(4, 13),
|
|
||||||
(4, 14),
|
|
||||||
(4, 15),
|
|
||||||
(4, 16),
|
|
||||||
(4, 17),
|
|
||||||
(4, 18),
|
|
||||||
(4, 19),
|
|
||||||
(4, 20),
|
|
||||||
(4, 21),
|
|
||||||
(4, 22),
|
|
||||||
(4, 23),
|
|
||||||
(4, 24),
|
|
||||||
(4, 25),
|
|
||||||
(5, 26),
|
|
||||||
(5, 27),
|
|
||||||
(5, 28),
|
|
||||||
(6, 29),
|
|
||||||
(6, 30),
|
|
||||||
(6, 31),
|
|
||||||
(6, 32),
|
|
||||||
(6, 33),
|
|
||||||
(6, 34),
|
|
||||||
(6, 35),
|
|
||||||
(6, 36),
|
|
||||||
(6, 37),
|
|
||||||
(6, 38),
|
|
||||||
(6, 39),
|
|
||||||
(7, 40),
|
|
||||||
(7, 41),
|
|
||||||
(7, 42),
|
|
||||||
(7, 43),
|
|
||||||
(7, 44),
|
|
||||||
(7, 45),
|
|
||||||
(7, 46),
|
|
||||||
(7, 47),
|
|
||||||
(7, 48),
|
|
||||||
(8, 49),
|
|
||||||
(8, 50),
|
|
||||||
(8, 51),
|
|
||||||
(8, 52),
|
|
||||||
(8, 53),
|
|
||||||
(8, 54),
|
|
||||||
(8, 55),
|
|
||||||
(8, 56),
|
|
||||||
(8, 57),
|
|
||||||
(8, 58),
|
|
||||||
(8, 59),
|
|
||||||
(8, 60),
|
|
||||||
(8, 61),
|
|
||||||
(8, 62),
|
|
||||||
(8, 63),
|
|
||||||
(8, 64),
|
|
||||||
(8, 65),
|
|
||||||
(8, 66),
|
|
||||||
(8, 67),
|
|
||||||
(8, 68),
|
|
||||||
(9, 69),
|
|
||||||
(9, 70),
|
|
||||||
(9, 71),
|
|
||||||
(10, 72),
|
|
||||||
(10, 73),
|
|
||||||
(10, 74),
|
|
||||||
(10, 75),
|
|
||||||
(10, 76),
|
|
||||||
(10, 77),
|
|
||||||
(10, 78),
|
|
||||||
(10, 79),
|
|
||||||
(10, 80),
|
|
||||||
(10, 81),
|
|
||||||
(11, 82),
|
|
||||||
(11, 83),
|
|
||||||
(11, 84),
|
|
||||||
(11, 85),
|
|
||||||
(11, 86),
|
|
||||||
(11, 87),
|
|
||||||
(11, 88),
|
|
||||||
(11, 89),
|
|
||||||
(11, 90),
|
|
||||||
(11, 91),
|
|
||||||
(11, 92),
|
|
||||||
(11, 93),
|
|
||||||
(11, 94),
|
|
||||||
(11, 95),
|
|
||||||
(12, 96),
|
|
||||||
(12, 97),
|
|
||||||
(12, 98),
|
|
||||||
(12, 81),
|
|
||||||
(12, 99),
|
|
||||||
(13, 100),
|
|
||||||
(13, 101),
|
|
||||||
(13, 102),
|
|
||||||
(13, 103),
|
|
||||||
(13, 104),
|
|
||||||
(13, 105),
|
|
||||||
(13, 106),
|
|
||||||
(14, 107),
|
|
||||||
(14, 108),
|
|
||||||
(14, 109),
|
|
||||||
(14, 110),
|
|
||||||
(14, 111),
|
|
||||||
(14, 112),
|
|
||||||
(14, 113),
|
|
||||||
(14, 114),
|
|
||||||
(14, 115),
|
|
||||||
(14, 116),
|
|
||||||
(14, 117),
|
|
||||||
(15, 13),
|
|
||||||
(15, 118),
|
|
||||||
(15, 119),
|
|
||||||
(15, 120),
|
|
||||||
(15, 23),
|
|
||||||
(15, 121),
|
|
||||||
(16, 122),
|
|
||||||
(16, 39),
|
|
||||||
(16, 123),
|
|
||||||
(16, 124),
|
|
||||||
(16, 125),
|
|
||||||
(16, 126),
|
|
||||||
(17, 127),
|
|
||||||
(17, 128),
|
|
||||||
(17, 129),
|
|
||||||
(17, 130),
|
|
||||||
(17, 131),
|
|
||||||
(18, 132),
|
|
||||||
(18, 133),
|
|
||||||
(19, 134),
|
|
||||||
(19, 135),
|
|
||||||
(19, 136),
|
|
||||||
(19, 137),
|
|
||||||
(19, 138),
|
|
||||||
(19, 139),
|
|
||||||
(19, 140),
|
|
||||||
(19, 141),
|
|
||||||
(19, 142),
|
|
||||||
(20, 141),
|
|
||||||
(20, 143),
|
|
||||||
(20, 144),
|
|
||||||
(20, 145),
|
|
||||||
(20, 146),
|
|
||||||
(20, 142),
|
|
||||||
(20, 147),
|
|
||||||
(20, 148),
|
|
||||||
(20, 149),
|
|
||||||
(20, 95),
|
|
||||||
(20, 150),
|
|
||||||
(20, 151),
|
|
||||||
(20, 152),
|
|
||||||
(20, 153),
|
|
||||||
(21, 154),
|
|
||||||
(21, 155),
|
|
||||||
(21, 156),
|
|
||||||
(21, 157),
|
|
||||||
(21, 158),
|
|
||||||
(21, 159),
|
|
||||||
(21, 160),
|
|
||||||
(21, 161),
|
|
||||||
(21, 162),
|
|
||||||
(21, 163),
|
|
||||||
(21, 164),
|
|
||||||
(21, 165),
|
|
||||||
(21, 166),
|
|
||||||
(21, 167),
|
|
||||||
(21, 168),
|
|
||||||
(21, 169),
|
|
||||||
(21, 170),
|
|
||||||
(22, 171),
|
|
||||||
(22, 172),
|
|
||||||
(22, 173),
|
|
||||||
(22, 174),
|
|
||||||
(23, 175),
|
|
||||||
(23, 176),
|
|
||||||
(23, 177),
|
|
||||||
(23, 178),
|
|
||||||
(23, 179),
|
|
||||||
(23, 180),
|
|
||||||
(23, 181),
|
|
||||||
(24, 182),
|
|
||||||
(24, 183),
|
|
||||||
(25, 184),
|
|
||||||
(25, 185),
|
|
||||||
(25, 186),
|
|
||||||
(25, 187),
|
|
||||||
(25, 188),
|
|
||||||
(25, 189),
|
|
||||||
(25, 190),
|
|
||||||
(25, 191),
|
|
||||||
(25, 192),
|
|
||||||
(25, 193),
|
|
||||||
(25, 194),
|
|
||||||
(25, 195),
|
|
||||||
(25, 196),
|
|
||||||
(25, 197),
|
|
||||||
(25, 198),
|
|
||||||
(26, 199),
|
|
||||||
(26, 200),
|
|
||||||
(26, 201),
|
|
||||||
(26, 202),
|
|
||||||
(26, 203),
|
|
||||||
(26, 204),
|
|
||||||
(26, 205),
|
|
||||||
(26, 206),
|
|
||||||
(26, 207),
|
|
||||||
(26, 208),
|
|
||||||
(26, 209),
|
|
||||||
(26, 210),
|
|
||||||
(26, 211),
|
|
||||||
(26, 212),
|
|
||||||
(26, 213),
|
|
||||||
(27, 214),
|
|
||||||
(27, 215),
|
|
||||||
(27, 216),
|
|
||||||
(27, 44),
|
|
||||||
(27, 217),
|
|
||||||
(27, 218),
|
|
||||||
(27, 41),
|
|
||||||
(27, 219),
|
|
||||||
(27, 45),
|
|
||||||
(27, 220),
|
|
||||||
(28, 71),
|
|
||||||
(28, 70),
|
|
||||||
(30, 221),
|
|
||||||
(30, 222),
|
|
||||||
(30, 131),
|
|
||||||
(31, 223),
|
|
||||||
(31, 224),
|
|
||||||
(31, 225),
|
|
||||||
(32, 226),
|
|
||||||
(32, 85),
|
|
||||||
(32, 227),
|
|
||||||
(32, 228),
|
|
||||||
(32, 229),
|
|
||||||
(32, 230),
|
|
||||||
(32, 231),
|
|
||||||
(32, 232),
|
|
||||||
(32, 112),
|
|
||||||
(32, 233),
|
|
||||||
(32, 234),
|
|
||||||
(32, 235),
|
|
||||||
(33, 236),
|
|
||||||
(33, 237),
|
|
||||||
(33, 238),
|
|
||||||
(34, 239),
|
|
||||||
(34, 240),
|
|
||||||
(34, 241),
|
|
||||||
(34, 242),
|
|
||||||
(34, 243),
|
|
||||||
(34, 244),
|
|
||||||
(35, 245),
|
|
||||||
(35, 246),
|
|
||||||
(36, 247),
|
|
||||||
(36, 248),
|
|
||||||
(37, 249),
|
|
||||||
(37, 250),
|
|
||||||
(38, 251),
|
|
||||||
(38, 252),
|
|
||||||
(38, 253),
|
|
||||||
(39, 254),
|
|
||||||
(39, 255),
|
|
||||||
(39, 256),
|
|
||||||
(39, 257),
|
|
||||||
(39, 258),
|
|
||||||
(39, 259),
|
|
||||||
(39, 260),
|
|
||||||
(39, 261),
|
|
||||||
(39, 262),
|
|
||||||
(40, 263),
|
|
||||||
(40, 264),
|
|
||||||
(40, 265),
|
|
||||||
(41, 266),
|
|
||||||
(41, 267),
|
|
||||||
(42, 268),
|
|
||||||
(42, 269),
|
|
||||||
(42, 270),
|
|
||||||
(42, 271),
|
|
||||||
(42, 272),
|
|
||||||
(43, 190),
|
|
||||||
(43, 273),
|
|
||||||
(43, 274),
|
|
||||||
(43, 275),
|
|
||||||
(43, 276),
|
|
||||||
(43, 277),
|
|
||||||
(43, 278),
|
|
||||||
(43, 279),
|
|
||||||
(43, 280),
|
|
||||||
(43, 186),
|
|
||||||
(44, 281),
|
|
||||||
(44, 164),
|
|
||||||
(44, 282),
|
|
||||||
(44, 154),
|
|
||||||
(44, 283),
|
|
||||||
(44, 284),
|
|
||||||
(44, 285),
|
|
||||||
(44, 286),
|
|
||||||
(44, 287),
|
|
||||||
(44, 288),
|
|
||||||
(44, 289),
|
|
||||||
(44, 290),
|
|
||||||
(44, 291),
|
|
||||||
(45, 292),
|
|
||||||
(45, 293),
|
|
||||||
(45, 294),
|
|
||||||
(46, 295),
|
|
||||||
(47, 165),
|
|
||||||
(47, 296),
|
|
||||||
(47, 166),
|
|
||||||
(47, 297),
|
|
||||||
(47, 298),
|
|
||||||
(47, 299),
|
|
||||||
(47, 160),
|
|
||||||
(47, 300),
|
|
||||||
(48, 301),
|
|
||||||
(48, 302),
|
|
||||||
(48, 303),
|
|
||||||
(48, 304),
|
|
||||||
(48, 305),
|
|
||||||
(48, 306),
|
|
||||||
(48, 307),
|
|
||||||
(48, 308),
|
|
||||||
(49, 309),
|
|
||||||
(49, 310),
|
|
||||||
(49, 311),
|
|
||||||
(50, 312),
|
|
||||||
(50, 313),
|
|
||||||
(51, 314),
|
|
||||||
(52, 315),
|
|
||||||
(52, 316),
|
|
||||||
(52, 317),
|
|
||||||
(52, 318),
|
|
||||||
(53, 319),
|
|
||||||
(53, 320),
|
|
||||||
(54, 321),
|
|
||||||
(54, 322),
|
|
||||||
(54, 323),
|
|
||||||
(54, 324),
|
|
||||||
(54, 325),
|
|
||||||
(54, 326),
|
|
||||||
(54, 327),
|
|
||||||
(54, 328),
|
|
||||||
(55, 329),
|
|
||||||
(55, 330),
|
|
||||||
(55, 331),
|
|
||||||
(55, 332),
|
|
||||||
(55, 333),
|
|
||||||
(56, 334),
|
|
||||||
(56, 335),
|
|
||||||
(57, 336),
|
|
||||||
(57, 337),
|
|
||||||
(57, 338),
|
|
||||||
(57, 339),
|
|
||||||
(57, 340),
|
|
||||||
(57, 341),
|
|
||||||
(57, 342),
|
|
||||||
(57, 343),
|
|
||||||
(58, 344),
|
|
||||||
(58, 345),
|
|
||||||
(58, 238),
|
|
||||||
(58, 346),
|
|
||||||
(58, 347),
|
|
||||||
(58, 348),
|
|
||||||
(58, 349),
|
|
||||||
(58, 350),
|
|
||||||
(58, 351),
|
|
||||||
(58, 352),
|
|
||||||
(58, 353),
|
|
||||||
(58, 237),
|
|
||||||
(58, 354),
|
|
||||||
(59, 355),
|
|
||||||
(59, 356),
|
|
||||||
(59, 357),
|
|
||||||
(59, 358),
|
|
||||||
(59, 359),
|
|
||||||
(60, 360),
|
|
||||||
(60, 361),
|
|
||||||
(61, 362),
|
|
||||||
(61, 363),
|
|
||||||
(62, 364),
|
|
||||||
(62, 365);
|
|
||||||
|
|
||||||
ALTER TABLE `transcripts`
|
|
||||||
DROP COLUMN `execs`,
|
|
||||||
DROP COLUMN `analysts`;
|
|
||||||
|
|
||||||
DELETE FROM `transcripts` WHERE `id` = 1;
|
|
||||||
DELETE FROM `execs` WHERE `id` = 1;
|
|
||||||
DELETE FROM `analysts` WHERE `id` = 1;
|
|
||||||
DELETE FROM `execs_to_transcripts` WHERE `transcript_id` = 0;
|
|
||||||
DELETE FROM `analysts_to_transcripts` WHERE `transcript_id` = 0;
|
|
||||||
|
|
@ -1,208 +0,0 @@
|
||||||
INSERT INTO `execs_to_transcripts` (`exec_id`, `transcript_id`) VALUES
|
|
||||||
(1, 0),
|
|
||||||
(2, 1),
|
|
||||||
(2, 2),
|
|
||||||
(2, 3),
|
|
||||||
(2, 4),
|
|
||||||
(3, 5),
|
|
||||||
(3, 6),
|
|
||||||
(3, 7),
|
|
||||||
(4, 8),
|
|
||||||
(4, 9),
|
|
||||||
(4, 10),
|
|
||||||
(5, 11),
|
|
||||||
(5, 12),
|
|
||||||
(5, 13),
|
|
||||||
(5, 14),
|
|
||||||
(6, 15),
|
|
||||||
(6, 16),
|
|
||||||
(6, 17),
|
|
||||||
(6, 18),
|
|
||||||
(6, 19),
|
|
||||||
(7, 20),
|
|
||||||
(7, 21),
|
|
||||||
(7, 22),
|
|
||||||
(7, 23),
|
|
||||||
(8, 24),
|
|
||||||
(8, 25),
|
|
||||||
(8, 26),
|
|
||||||
(9, 27),
|
|
||||||
(9, 28),
|
|
||||||
(9, 29),
|
|
||||||
(9, 30),
|
|
||||||
(10, 31),
|
|
||||||
(10, 32),
|
|
||||||
(11, 33),
|
|
||||||
(11, 34),
|
|
||||||
(11, 35),
|
|
||||||
(12, 36),
|
|
||||||
(12, 37),
|
|
||||||
(12, 38),
|
|
||||||
(12, 39),
|
|
||||||
(13, 40),
|
|
||||||
(13, 41),
|
|
||||||
(13, 42),
|
|
||||||
(14, 43),
|
|
||||||
(14, 44),
|
|
||||||
(14, 45),
|
|
||||||
(14, 46),
|
|
||||||
(14, 47),
|
|
||||||
(14, 48),
|
|
||||||
(14, 49),
|
|
||||||
(14, 50),
|
|
||||||
(14, 51),
|
|
||||||
(15, 52),
|
|
||||||
(15, 53),
|
|
||||||
(15, 54),
|
|
||||||
(16, 55),
|
|
||||||
(16, 56),
|
|
||||||
(16, 57),
|
|
||||||
(17, 58),
|
|
||||||
(17, 59),
|
|
||||||
(17, 60),
|
|
||||||
(17, 61),
|
|
||||||
(18, 62),
|
|
||||||
(18, 63),
|
|
||||||
(19, 64),
|
|
||||||
(19, 65),
|
|
||||||
(19, 66),
|
|
||||||
(20, 67),
|
|
||||||
(20, 68),
|
|
||||||
(21, 69),
|
|
||||||
(21, 70),
|
|
||||||
(21, 71),
|
|
||||||
(22, 72),
|
|
||||||
(22, 73),
|
|
||||||
(22, 74),
|
|
||||||
(23, 75),
|
|
||||||
(23, 76),
|
|
||||||
(23, 77),
|
|
||||||
(23, 78),
|
|
||||||
(24, 79),
|
|
||||||
(24, 80),
|
|
||||||
(24, 81),
|
|
||||||
(25, 82),
|
|
||||||
(25, 83),
|
|
||||||
(25, 84),
|
|
||||||
(26, 85),
|
|
||||||
(26, 86),
|
|
||||||
(26, 87),
|
|
||||||
(26, 88),
|
|
||||||
(27, 89),
|
|
||||||
(27, 90),
|
|
||||||
(27, 91),
|
|
||||||
(27, 92),
|
|
||||||
(27, 93),
|
|
||||||
(28, 94),
|
|
||||||
(28, 95),
|
|
||||||
(28, 96),
|
|
||||||
(29, 97),
|
|
||||||
(29, 98),
|
|
||||||
(29, 99),
|
|
||||||
(30, 100),
|
|
||||||
(30, 101),
|
|
||||||
(31, 102),
|
|
||||||
(31, 103),
|
|
||||||
(31, 104),
|
|
||||||
(31, 105),
|
|
||||||
(32, 106),
|
|
||||||
(32, 107),
|
|
||||||
(33, 108),
|
|
||||||
(33, 109),
|
|
||||||
(34, 110),
|
|
||||||
(34, 111),
|
|
||||||
(35, 112),
|
|
||||||
(35, 113),
|
|
||||||
(35, 114),
|
|
||||||
(36, 115),
|
|
||||||
(36, 116),
|
|
||||||
(36, 117),
|
|
||||||
(36, 118),
|
|
||||||
(37, 119),
|
|
||||||
(37, 120),
|
|
||||||
(37, 121),
|
|
||||||
(38, 122),
|
|
||||||
(38, 123),
|
|
||||||
(39, 124),
|
|
||||||
(39, 125),
|
|
||||||
(39, 126),
|
|
||||||
(40, 127),
|
|
||||||
(40, 128),
|
|
||||||
(40, 129),
|
|
||||||
(41, 130),
|
|
||||||
(41, 131),
|
|
||||||
(41, 132),
|
|
||||||
(41, 133),
|
|
||||||
(42, 134),
|
|
||||||
(42, 135),
|
|
||||||
(42, 136),
|
|
||||||
(42, 137),
|
|
||||||
(42, 138),
|
|
||||||
(42, 139),
|
|
||||||
(43, 140),
|
|
||||||
(43, 141),
|
|
||||||
(43, 142),
|
|
||||||
(44, 143),
|
|
||||||
(44, 144),
|
|
||||||
(44, 145),
|
|
||||||
(45, 146),
|
|
||||||
(45, 147),
|
|
||||||
(45, 148),
|
|
||||||
(46, 149),
|
|
||||||
(46, 150),
|
|
||||||
(46, 151),
|
|
||||||
(47, 152),
|
|
||||||
(47, 153),
|
|
||||||
(47, 154),
|
|
||||||
(47, 155),
|
|
||||||
(48, 156),
|
|
||||||
(48, 157),
|
|
||||||
(48, 158),
|
|
||||||
(49, 159),
|
|
||||||
(49, 160),
|
|
||||||
(49, 161),
|
|
||||||
(50, 162),
|
|
||||||
(50, 163),
|
|
||||||
(51, 164),
|
|
||||||
(51, 165),
|
|
||||||
(52, 166),
|
|
||||||
(52, 167),
|
|
||||||
(52, 168),
|
|
||||||
(53, 169),
|
|
||||||
(53, 170),
|
|
||||||
(53, 171),
|
|
||||||
(54, 172),
|
|
||||||
(54, 173),
|
|
||||||
(54, 174),
|
|
||||||
(54, 175),
|
|
||||||
(54, 176),
|
|
||||||
(54, 177),
|
|
||||||
(55, 178),
|
|
||||||
(55, 179),
|
|
||||||
(55, 180),
|
|
||||||
(56, 181),
|
|
||||||
(56, 182),
|
|
||||||
(57, 183),
|
|
||||||
(57, 184),
|
|
||||||
(57, 185),
|
|
||||||
(58, 186),
|
|
||||||
(58, 187),
|
|
||||||
(58, 188),
|
|
||||||
(58, 189),
|
|
||||||
(58, 190),
|
|
||||||
(58, 191),
|
|
||||||
(58, 192),
|
|
||||||
(58, 193),
|
|
||||||
(58, 194),
|
|
||||||
(58, 195),
|
|
||||||
(59, 196),
|
|
||||||
(59, 197),
|
|
||||||
(59, 198),
|
|
||||||
(60, 199),
|
|
||||||
(60, 200),
|
|
||||||
(60, 201),
|
|
||||||
(61, 202),
|
|
||||||
(61, 203),
|
|
||||||
(62, 204),
|
|
||||||
(62, 205),
|
|
||||||
(62, 206);
|
|
||||||
File diff suppressed because one or more lines are too long
|
|
@ -1,110 +0,0 @@
|
||||||
-- phpMyAdmin SQL Dump
|
|
||||||
-- version 4.0.10.14
|
|
||||||
-- http://www.phpmyadmin.net
|
|
||||||
--
|
|
||||||
-- Host: localhost:3306
|
|
||||||
-- Generation Time: Dec 26, 2016 at 02:39 PM
|
|
||||||
-- Server version: 5.5.52-cll-lve
|
|
||||||
-- PHP Version: 5.6.20
|
|
||||||
|
|
||||||
SET SQL_MODE = "NO_AUTO_VALUE_ON_ZERO";
|
|
||||||
SET time_zone = "+00:00";
|
|
||||||
|
|
||||||
|
|
||||||
/*!40101 SET @OLD_CHARACTER_SET_CLIENT=@@CHARACTER_SET_CLIENT */;
|
|
||||||
/*!40101 SET @OLD_CHARACTER_SET_RESULTS=@@CHARACTER_SET_RESULTS */;
|
|
||||||
/*!40101 SET @OLD_COLLATION_CONNECTION=@@COLLATION_CONNECTION */;
|
|
||||||
/*!40101 SET NAMES utf8 */;
|
|
||||||
|
|
||||||
--
|
|
||||||
-- Database: `bengoldsworthy`
|
|
||||||
--
|
|
||||||
|
|
||||||
-- --------------------------------------------------------
|
|
||||||
|
|
||||||
--
|
|
||||||
-- Table structure for table `transcripts`
|
|
||||||
--
|
|
||||||
|
|
||||||
CREATE TABLE IF NOT EXISTS `transcripts` (
|
|
||||||
`id` int(11) NOT NULL AUTO_INCREMENT,
|
|
||||||
`title` varchar(255) DEFAULT NULL,
|
|
||||||
`company` varchar(255) DEFAULT NULL,
|
|
||||||
`execs` varchar(255) DEFAULT NULL,
|
|
||||||
`analysts` varchar(255) DEFAULT NULL,
|
|
||||||
`transcript` text,
|
|
||||||
PRIMARY KEY (`id`),
|
|
||||||
UNIQUE KEY `id` (`id`)
|
|
||||||
) ENGINE=MyISAM DEFAULT CHARSET=latin1 AUTO_INCREMENT=63 ;
|
|
||||||
|
|
||||||
--
|
|
||||||
-- Dumping data for table `transcripts`
|
|
||||||
--
|
|
||||||
|
|
||||||
INSERT INTO `transcripts` (`id`, `execs`, `analysts`) VALUES
|
|
||||||
(1, '0', '0'),
|
|
||||||
(2, '1;2;3;4', '1'),
|
|
||||||
(3, '5;6;7', '2;3;4;5;6;7;8;9;10'),
|
|
||||||
(4, '8;9;10', '11;12;13;14;15;16;17;18;19;20;21;22;23;24;25'),
|
|
||||||
(5, '11;12;13;14', '26;27;28'),
|
|
||||||
(6, '15;16;17;18;19', '29;30;31;32;33;34;35;36;37;38;39'),
|
|
||||||
(7, '20;21;22;23', '40;41;42;43;44;45;46;47;48'),
|
|
||||||
(8, '24;25;26', '49;50;51;52;53;54;55;56;57;58;59;60;61;62;63;64;65;66;67;68'),
|
|
||||||
(9, '27;28;29;30', '69;70;71'),
|
|
||||||
(10, '31;32', '72;73;74;75;76;77;78;79;80;81'),
|
|
||||||
(11, '33;34;35', '82;83;84;85;86;87;88;89;90;91;92;93;94;95'),
|
|
||||||
(12, '36;37;38;39', '96;97;98;81;99'),
|
|
||||||
(13, '40;41;42', '100;101;102;103;104;105;106'),
|
|
||||||
(14, '43;44;45;46;47;48;49;50;51', '107;108;109;110;111;112;113;114;115;116;117'),
|
|
||||||
(15, '52;53;54', '13;118;119;120;23;121'),
|
|
||||||
(16, '55;56;57', '122;39;123;124;125;126'),
|
|
||||||
(17, '58;59;60;61', '127;128;129;130;131'),
|
|
||||||
(18, '62;63', '132;133'),
|
|
||||||
(19, '64;65;66', '134;135;136;137;138;139;140;141;142'),
|
|
||||||
(20, '67;68', '141;143;144;145;146;142;147;148;149;95;150;151;152;153'),
|
|
||||||
(21, '69;70;71', '154;155;156;157;158;159;160;161;162;163;164;165;166;167;168;169;170'),
|
|
||||||
(22, '72;73;74', '171;172;173;174'),
|
|
||||||
(23, '75;76;77;78', '175;176;177;178;179;180;181'),
|
|
||||||
(24, '79;80;81', '182;183'),
|
|
||||||
(25, '82;83;84', '184;185;186;187;188;189;190;191;192;193;194;195;196;197;198'),
|
|
||||||
(26, '85;86;87;88', '199;200;201;202;203;204;205;206;207;208;209;210;211;212;213'),
|
|
||||||
(27, '89;90;91;92;93', '214;215;216;44;217;218;41;219;45;220'),
|
|
||||||
(28, '94;95;96', '71;70'),
|
|
||||||
(29, '97;98;99', ''),
|
|
||||||
(30, '100;101', '221;222;131'),
|
|
||||||
(31, '102;103;104;105', '223;224;225'),
|
|
||||||
(32, '106;107', '226;85;227;228;229;230;231;232;112;233;234;235'),
|
|
||||||
(33, '108;109', '236;237;238'),
|
|
||||||
(34, '110;111', '239;240;241;242;243;244'),
|
|
||||||
(35, '112;113;114', '245;246'),
|
|
||||||
(36, '115;116;117;118', '247;248'),
|
|
||||||
(37, '119;120;121', '249;250'),
|
|
||||||
(38, '122;123', '251;252;253'),
|
|
||||||
(39, '124;125;126', '254;255;256;257;258;259;260;261;262'),
|
|
||||||
(40, '127;128;129', '263;264;265'),
|
|
||||||
(41, '130;131;132;133', '266;267'),
|
|
||||||
(42, '134;135;136;137;138;139', '268;269;270;271;272'),
|
|
||||||
(43, '140;141;142', '190;273;274;275;276;277;278;279;280;186'),
|
|
||||||
(44, '143;144;145', '281;164;282;154;283;284;285;286;287;288;289;290;291'),
|
|
||||||
(45, '146;147;148', '292;293;294'),
|
|
||||||
(46, '149;150;151', '295'),
|
|
||||||
(47, '152;153;154;155', '165;296;166;297;298;299;160;300'),
|
|
||||||
(48, '156;157;158', '301;302;303;304;305;306;307;308'),
|
|
||||||
(49, '159;160;161', '309;310;311'),
|
|
||||||
(50, '162;163', '312;313'),
|
|
||||||
(51, '164;165', '314'),
|
|
||||||
(52, '166;167;168', '315;316;317;318'),
|
|
||||||
(53, '169;170;171', '319;320'),
|
|
||||||
(54, '172;173;174;175;176;177', '321;322;323;324;325;326;327;328'),
|
|
||||||
(55, '178;179;180', '329;330;331;332;333'),
|
|
||||||
(56, '181;182', '334;335'),
|
|
||||||
(57, '183;184;185', '336;337;338;339;340;341;342;343'),
|
|
||||||
(58, '186;187;188;189;190;191;192;193;194;195', '344;345;238;346;347;348;349;350;351;352;353;237;354'),
|
|
||||||
(59, '196;197;198', '355;356;357;358;359'),
|
|
||||||
(60, '199;200;201', '360;361'),
|
|
||||||
(61, '202;203', '362;363'),
|
|
||||||
(62, '204;205;206', '364;365');
|
|
||||||
|
|
||||||
/*!40101 SET CHARACTER_SET_CLIENT=@OLD_CHARACTER_SET_CLIENT */;
|
|
||||||
/*!40101 SET CHARACTER_SET_RESULTS=@OLD_CHARACTER_SET_RESULTS */;
|
|
||||||
/*!40101 SET COLLATION_CONNECTION=@OLD_COLLATION_CONNECTION */;
|
|
||||||
Reference in a new issue