2016-02-20 19:42:37 -05:00
{
"cells": [
{
"cell_type": "markdown",
2021-12-24 13:29:47 -05:00
"metadata": {},
2016-02-20 19:42:37 -05:00
"source": [
2017-02-25 13:23:51 -05:00
"# Demonstration of `pdfplumber`'s table-extraction options\n",
2016-02-20 19:42:37 -05:00
"\n",
2017-02-25 13:23:51 -05:00
"This notebook uses a [report from the FBI's National Instant Criminal Background Check System](../pdfs/background-checks.pdf)."
2016-02-20 19:42:37 -05:00
]
},
{
"cell_type": "markdown",
2021-12-24 13:29:47 -05:00
"metadata": {},
2016-02-20 19:42:37 -05:00
"source": [
2016-03-06 18:30:41 -05:00
"### Import `pdfplumber`"
2016-02-20 19:42:37 -05:00
]
},
{
"cell_type": "code",
"execution_count": 1,
2022-05-31 15:50:50 -04:00
"metadata": {
"execution": {
2022-07-18 10:48:11 -04:00
"iopub.execute_input": "2022-07-18T14:47:50.653979Z",
"iopub.status.busy": "2022-07-18T14:47:50.653590Z",
"iopub.status.idle": "2022-07-18T14:47:50.688352Z",
"shell.execute_reply": "2022-07-18T14:47:50.688022Z"
2022-05-31 15:50:50 -04:00
}
},
2021-12-24 13:29:47 -05:00
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
2024-08-07 15:50:10 -04:00
"0.11.3\n"
2021-12-24 13:29:47 -05:00
]
}
],
2016-02-20 19:42:37 -05:00
"source": [
2021-12-24 13:29:47 -05:00
"import pdfplumber\n",
"print(pdfplumber.__version__)"
2016-02-20 19:42:37 -05:00
]
},
{
"cell_type": "markdown",
2021-12-24 13:29:47 -05:00
"metadata": {},
2016-02-20 19:42:37 -05:00
"source": [
"### Load the PDF"
]
},
{
"cell_type": "code",
"execution_count": 2,
2022-05-31 15:50:50 -04:00
"metadata": {
"execution": {
2022-07-18 10:48:11 -04:00
"iopub.execute_input": "2022-07-18T14:47:50.690282Z",
"iopub.status.busy": "2022-07-18T14:47:50.690148Z",
"iopub.status.idle": "2022-07-18T14:47:50.693123Z",
"shell.execute_reply": "2022-07-18T14:47:50.692800Z"
2022-05-31 15:50:50 -04:00
}
},
2016-02-20 19:42:37 -05:00
"outputs": [],
"source": [
2016-03-09 07:21:17 -05:00
"pdf = pdfplumber.open(\"../pdfs/background-checks.pdf\")"
2016-02-20 19:42:37 -05:00
]
},
{
"cell_type": "markdown",
2021-12-24 13:29:47 -05:00
"metadata": {},
2016-02-20 19:42:37 -05:00
"source": [
2016-03-06 18:30:41 -05:00
"### Get the first page"
2016-02-20 19:42:37 -05:00
]
},
{
"cell_type": "code",
"execution_count": 3,
2022-05-31 15:50:50 -04:00
"metadata": {
"execution": {
2022-07-18 10:48:11 -04:00
"iopub.execute_input": "2022-07-18T14:47:50.694914Z",
"iopub.status.busy": "2022-07-18T14:47:50.694698Z",
"iopub.status.idle": "2022-07-18T14:47:50.697918Z",
"shell.execute_reply": "2022-07-18T14:47:50.697541Z"
2022-05-31 15:50:50 -04:00
}
},
2016-02-20 19:42:37 -05:00
"outputs": [],
"source": [
2017-02-25 13:23:51 -05:00
"p0 = pdf.pages[0]"
]
},
{
"cell_type": "code",
"execution_count": 4,
2022-05-31 15:50:50 -04:00
"metadata": {
"execution": {
2022-07-18 10:48:11 -04:00
"iopub.execute_input": "2022-07-18T14:47:50.699690Z",
"iopub.status.busy": "2022-07-18T14:47:50.699510Z",
"iopub.status.idle": "2022-07-18T14:47:50.932605Z",
"shell.execute_reply": "2022-07-18T14:47:50.932325Z"
2022-05-31 15:50:50 -04:00
}
},
2017-02-25 13:23:51 -05:00
"outputs": [
{
"data": {
2024-08-07 15:50:10 -04:00
"image/png": "iVBORw0KGgoAAAANSUhEUgAAA/AAAAJkCAMAAACBATrdAAADAFBMVEX+/v7L/v7t6+EAAABoaGh3d3eZmZnX19eGhoa3t7fb6fHIyMinp6f+/gBYWFjq8N386Njx29r+/stJSUlVaWp4l5e55+eGqKhqhoY1NzeTuLhjentHWVmcxMSq1dU6SEhzi4xcc3P+AQEmKCiiy8srNjaw3d1NYWGNsbEXGRnS5umGengcIyO4uACnmJaWiYa5qKWHhXl5dWloZlrjhYVkW1jYxsWoqAB0amh4eACntbpnZwDW1gCZpamJlJjF1NqWlgDYtrZXVEr6FhaGhgDJt7Xm5gC1t6nzNja2xMq0pprGxgCTnKINERGWlnjWxbnO6OrU2cjglJT2JibFtanq1NOEjZLmeHikppnpaGjcqKjEybrn57ipqYbk6deXlovn1MmjrLNYWABmbXHV1arN2uJ1fYGEhGnCy9XizsuRhn3tV1e6upRSS0nS3ObwRUWzu8Rzc1ve49FFRDyNkIU2Ni1DPDxtcGcxLSzM0sFJSQCjlovDw5utvcFYXWHfnZ3d3bC8zNF+gXedoZTKyqEiIB1iYk5KTVG+wrNOUEorLjCtsaOxsY3vTk4UFAggHRzw3c04OAAQDg6/8PBeYVnAr64vMCyerbDb4M8uLgB/oKAdHyAPEA7K0L+Bd28fIB2wn54AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAC5yDhfAAAACXBIWXMAAAsTAAALEwEAmpwYAAC3RElEQVR4nO39h4PjxpE2DoM9HQA1BEkmQJAEyRnuZK200SutVjlnybIlWbZsBed8zr47Xw7v5XvzL3z5T/2qGiCHJFJxiKXInS6NljNEodHo7gdo9IOqx3GsWbNmzZo1a9asWbNmzZo1a9asWbNmzZo1a9asWbNmzZo1a9asWbNmzZo1a9asWbNmzZo1a9asWbNmzZo1a9asWbNmzZo1a9asWbNmzZo1a9asWbNmzZo1a9asWbNmzZo1a9asWSuwHWvWrG2JNQH4ljVr1rbDGkC8Bbw1a9tiFvDWrF0gs4C/l6zTyf+2gTb6oitwgc0Cfmvt7bcNqA970Ish/NIJH9nZOY3wu/Bt+K2b+SXZck1rd3ep8mOz1+FBfku0s/TlpLMTZb8Nd7NSy0o5c7XWvFnAb629vWMAfHiYAf5wZxDvHe70Ae87gyja3UlSv2Qn6YJFrWS8VPnxznG3C5eOPCxXAfxw55EkHr/9iAX8F2MW8FtrcBdHZEwAP9iJ4a/OKUDp9BC3P3Ka+iXzwDoy/2RfdUadmW/Nv1PnGK8drX4Kv6PJNBw3z0D1aPZzlENwtlenM0XxzmnHlB2aUibbRzP+Z4Dv2Ll/42YBv7X29uARBN4E8G+nE/ZhjFBHUI36qd8U8L1ea29nd+eRVny6s7MPYMLPnUEr/TbaCXd2Hjk43NkZp+4p4Id4HRnj5B5g2IGth30D1b30IrPzSG8/KzUCn32Yqh9iRXq78DfuBnsf7O+AQ4rivrksQeFD2A6775xARQ/T0mFmsnOaGMAfvL3fwYNN6mKtKbOA31p7+3iIk/oM8KOd7nQLQG+3O332TnaiGKyTAv6RbnSws9uPcCYA4B0OAHLm22jnkT7sOOgPsgsETOn3uuO3D839eBQ/Av28fxoND08R8AbvxztJf3dnP93/ZKcHpZ7OAv6RqH/6Nk464ug0A3x3Z3RWy9OTGHd45DSCYoZQ0fBgvNMHwI8A7wD/k9ExfG2tSbOA31p7+xjuvNEE8CezT759vHfuZ9DKFu1GKeD7eF/uIIjjTgIeB7Cf+TbCfxDVo8lNON0PZgB9XA0Y73TM3X4UjqKdxNx6zSKCATzsuoulRlihKeD7CPDOCYI2ezKAY08rabYnuB03wXFO4VCtMO7shKeHUNbgbdh+sslswzaaBfzWGgAeANrJAH80WaPLbAjz6fS3ZGcEz9CdbEqPDwE7j4DBjKCT7O7vGMAbrB6l04XOFPAAuE5kEB/uwuy/M3l2j7KpukHq7n66v1k46MCF4AzwptRO19Sj6A5vrjtH3Z1TrM5hZ2fPfN+B0rGI4c7O24P+XWzAC2kW8FtrCHiY1E+e4dM1+9be/nC4a6bz5g7aWnyG75iJOdrw6O23B934DPCddDo+WTRLn+Hhzo2T8zAe4zx+Avjk8O0OAB6vDIMM8I+kgE9SwB/OAv7orNDJM/zu7gzgx1id+Azwh+N0ATLa3bEr9g2bBfzWGgIel9PmV+nf3ul08KaMWEsf43OAH+Bd9mAw7OJc+wRgVgn4R05bh/C8jw8CBq0wrwfXA7y+mOOcZoDPpvQnrR4if+cM8GY2H02Qe5qt0p8BPjarD8d7aXGPDPD4+3A9SRKsy2BdzXlBzAJ+a80AHib1Ex5+fyeMk0fwNjnY2Y3gjryf+uUAP9w5jPv7p4D18dHJKcCtDPDHe93uIdyzBzv9URdXAXB57fAR45oA+Mc7u91HdjLA93d6JxHCOdzp9ndnAI8ziviRGR6+Gw+wsAngodTuARwCiguH4c7QrNLD/t2d6GBvZinSWhNmAb+19jaiHNGbAf7oePqm3Tjj24xNAb/byxh0pON2h4ZV2+k+MgPNDPDprPvErNntw3121IPPPcAhEmi7w9T1EbwJP7IzgEeKtNTobZhtHKXc3e5gegfvtI7gi8F0bh7jm3a9g7NneFPqIwhs5AX30gsOTuoHMydhrSGzgL+XbOZd+qMqv+nbNdVeBQXPvaO/h88Mj5y9sHtU5FTwRf59mpL62BdvGjcLeGvntv3Tk4PQLqttlVnAWzu3YRzMqX3I3iqzgLe2itn3YrbMLOCtWbtAZgFvzdoFMgv4C2hfmfsg+9fYwzn3h4sdrX2BZgF/8ez5S6/hh7r0ZZq/uvTVeqdvXLp06acI9W+8eOlrynz19UsPm7/RLPY3xCzgL549f8lAfQnA1zt+9dJPv/rTSz9F4P/0r198Eb/68iUD+K+9+A2wFaprrUmzgL94BoC/pCY4/spjeDv+ipmGm9uweixzMxvwD3B8+LF0nv7wYw+njo8tlPm1r+E/l1oPI+ifhynBV167lAL+kgX7JpkF/MWz5y99Ayf1BvBfB1i++HzrNbwnf/PSN1uPfe2Suf+/9rUXL136k6/Cxr8GR/zy63BZ+Cl8/snDcO9+7dKL82V+E68AAPjnLz0PvwDq1Ytf/TIC/rFLX33+m2r9J2mt2CzgL549f+mb3wBQI+D/+tJr3/zyi5ce/iri9D8utVovvvjV5/8EgP/apZ8+/x8A/edffA0cL33DfPmNS994DKbsAPgXv1zwWP/8pf+AAhH5L/4J/m0A/9eXLqWXEGsbYRbwF88A8HAzVgj41xCTX7305a8AjL8Ck+9vGmhe+hPYgDfnP8EpAAAeF/kAxOYTLgtfhgLy9tilF/Hmj3fzr5lVQQP4r1/6+sPf/NqlxUcAa1+QWcBfPEPAP3/pNQT8i/jorQDpP70EuH8MZ/gvvvjipa/BlD6b8wNqFU7nYdav0o2XHktRDfb1r33ta5NH9G9eevGxlilk7g5v7Kv2Fr8pZgF/8QwBj6vpZ4D/On6HGP/6pW98FeyvW6+9Ngt4hOuLX4M7PW786sNTJH/5tdde+3r661+nZFz2DG+uAsbtYbP0d+nr6z9Na0VmAX/xzAAeJvWA4z/Be7WZob/4GsL6mwaZ//H1BcD/FP/4aesSXh6+/h9
2017-02-25 13:23:51 -05:00
"text/plain": [
2024-08-07 15:50:10 -04:00
"<pdfplumber.display.PageImage at 0x105cf6bd0>"
2017-02-25 13:23:51 -05:00
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"im = p0.to_image()\n",
"im"
2016-02-20 19:42:37 -05:00
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
2017-02-25 13:23:51 -05:00
"### What data would we get if we used the default settings?\n",
2016-03-06 18:30:41 -05:00
"\n",
2017-02-25 13:23:51 -05:00
"We can check by using `PageImage.debug_tablefinder()`:"
2016-02-20 19:42:37 -05:00
]
},
{
"cell_type": "code",
2017-02-25 13:23:51 -05:00
"execution_count": 5,
2022-05-31 15:50:50 -04:00
"metadata": {
"execution": {
2022-07-18 10:48:11 -04:00
"iopub.execute_input": "2022-07-18T14:47:50.934494Z",
"iopub.status.busy": "2022-07-18T14:47:50.934380Z",
"iopub.status.idle": "2022-07-18T14:47:51.175271Z",
"shell.execute_reply": "2022-07-18T14:47:51.174997Z"
2022-05-31 15:50:50 -04:00
}
},
2017-02-25 13:23:51 -05:00
"outputs": [
{
"data": {
2024-08-07 15:50:10 -04:00
"image/png": "iVBORw0KGgoAAAANSUhEUgAAA/AAAAJkCAMAAACBATrdAAADAFBMVEXMzP7+/v6jzP69vOf7AgPzKzbIAApWVohmZpiVlcd3d6mmptiGhrjrKzZISHorK/63u/LHtuDMzDK8weOuvvQ0CdP/NzfMzNVFVYg1AMh+frGtruArAMp3lcdTZ5o2N2leXpCcw/VtbqBqhbcnKFoBAjRbc6U5R3mTuOqEpddjeaydntAAAACNjsBOToE3N/89PnBNYZPMATMXGUpyir3zKwsqNWeNseN+ntCntemYvvCpmMcpLmCKruBzaZhxjsCXlzKWibbzKy1YbqA+ToCFeahLXpC4p9ccJFa3aJp4dphWVXpXVzJnZzKIiDJjWolkfrCoqDKIlMpnZYh3dzKIhqi3tzJSS3sXF+nEKFrlHSi1dqiZpNuXlbrHF0kZHlBISDKxiLoMDkC0t9uoqLi7VYcsLNNERGsAAMgNEUOlpcu4uMVpaWmdquCXl6rCNmjFtdvbEx2Rha6+RnhBO2vKDkByco3ExMqNkbdYWFizs7rm5uazpcuFhZytntChlLx3d3fAruDcGCOGhobY2NhtcJmwjsBdYIp9gageHvDpIiutstVWToCdocZ5bqCOm9FOUHw5OTKzfrAqKjKqqqqWlpaej8A+QG1ISEghHlAiIk0xLV7AQHLDLmDeFgsODuHAP3GwkMIAANO9ToAgHk/lHQtEPnC6XpBpX5BgYH7HH1EUFDLMBRC2bqCLfrA1NVwjIyMzL2HoIAsAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA0IffTAAAACXBIWXMAAAsTAAALEwEAmpwYAADK5UlEQVR4nO39iWMbx5UujrZRlGtVE4qtmTYQNAACxBpwSJGXpCRLskZr5EXyIjtOvCR25vrF2TOZyTLJ7HNn7p39ztzld7ffvr79/YXvnKoGCaCXKhBNGBT7mBbIrtPV1VX1oavq6zqf91xhhRV2bsz7ogtQWGGFLc8KwBdW2DmyAvCFFXaOrAB8YYWdIysAX1hh58gKwBdW2DmyAvCFFXaOrAB8YYWdIysAX1hh58gKwBdW2DmyAvCFFXaOrAB8YYWdIysAX1hh58gKwBdW2DmyAvCFFXaOrAB8YYWdIysAX1hh58gKwBdW2DmyAvCFFXaOzAr43/qt31pGOQorrLCFzY5WG+B/K/q/sMIKW3VzQKsF8L818W9hhRW2yuaCVivgf2Ntbe2ltcIKK2zVDXG6MOCfW7t48fLFwgorbNXt8sWFAQ9nr5XWS4UVVtjq2/rFtQXn8ID4lwq8F1bY2bDLLy26Sg9D+i/6JgobW68X/20F7Ubziy7BuTV4wheAP6O2ualB3d6HXwP45Vqwu7n5nQH81gw24bcw8gs3jZX29+fKf0+f9XI/njLYvDZvYXubg+i3Wh1ybUOunc3kL6XeZmfe3AtztQLwZ9c2NzWA2y8bwDfbm91Wp71ZK5WCze6gs7+5Y/zCzbCDVtrZmSv/vc27nTD4JAGWgxSoZtgR4A83d3f2gk92mwXgvwgrAH92DR6UT0rHgO9utuCva98BKO22MX131/iF08A60P9Eh3r9axNHm/hvbzze3sPvjlLNwO8getA38cRjwOszjs6/MYvg5o3oMs0x4Jub3+npvAME/Dj9WvRZws9jwPdulArL1wrAn13b7H6MwBsD/pNtfbS21yx9vIug6teM3xHgYUjf2dzf/LjU+s7m5kNA8B7MATa7JXO0swkTgd1+e1PPD0rHgG/hkMF8u/RehhNrGvDgXmp24Yz9h1Gug02T68v742vhaXB2/+HmZj0CvM4OrHUI6XD65i1AeTv67gL/7+xowPc/edjrwWmb95dWnefDCsCfXdsMDnFQHwH+xmZ4lNKC0X54NPcONzstsKYB4e6dQX9zvzbY3W02N9u1wy5ATh+Ff2odwH+tG31B4JD+TvDJy03IL+i3djdLzYe7ncOXd5sAeMR76e5mWNvffGjOvwW5dna/09SLCuZaH7dqu5+Umru7rcFuBPhw8+ipDWfdau1+pwTfTwPIplbagcsEmzUAPOId4F/r392sLas6z4cVgD+7BpALAEYR4MePTm21bvS0RYsW7foGhDUc/PfwS6HVwy+FPjxP9VH9z+5mE46YnMyi3Wa3Warh5H9ns3eon9dBf7AZ6kfv5jaO0R+aU7cx1wEW6AjwNT280COFWgT47ubRCr1O38H0DmbTLe3CYKMUtHqbwXcewkyj+0mt1NxbZbbhDFoB+LNrOKaGQX0E+IOJJzzaIYyPDbbCzX6vhzSdBiFOAjY//vjjXfDvhV0YNnfM0c4mzMNffhnn0GPAAyB7Axzz14J9GP33xnP3QTRU1yDWQ3o4rhcOAKwTgIejTzZ7IZajefSEPxp5dMz3zkG4uQvl2Xw4nrv3IHecnhzit03xgM/XCsCfXcNBNQzqx3P4zbo+2nl4eLivQfUkGg5Pz+EBfA+/o1ftDw8+2eyGLQ34ZoQ/nH9PAR7O2gSAfxy0AhzHjwG/8/CTXjQU6D40539sAB8awD+cBPzB8Sr9eCAy/kLQgN/B4rSOAf9yoL16g+3NyXFLYYtbAfiza3pxDZ7j06v0m5vXevhQRliah2kM8F2cR/e7Nf20vWUD/Me7pTau98NEoGae6y140vfhIYzD8FLzOxHg982Qfg8f+ZB0DHi9zj8Yr9J/R6/St+D0MeBbODZp3u2UdLF3uwh8/D7ZCbEs3WVW6bNvBeDPrmnAw6B+zMM/3Axa4S6iq7u534En8kPjFwP84Wa7VXu426xt3j/Y+w7ALQ3w3U4nbG/uQH61GyGuAny826kB+nFovwNewWb9zsebEeBrm/u3Oru71+DonVp9AvDNh7uD1njRDtw+DltdzGwM+Obud+704RJwYgATkb5epYfxSrjZ6XdmJiqFLWgF4M+ufaLps8Mx4EsHd2GevatxtfMds9qmLQZ4Tcvt69W7zc1wNzqqX5+bBPwtvWb3EBDX34fPDkAWSbt9Q8vh8kFpZ3ez225HuQ4+gdEGjt7RqbtvMmyB1wEc6B69abeHb9rhpCNKPyj1X97c/PiOeUHw444Z/cOgHkm/o5soLB8rAP9M2cS79AdZfuO3a5rZXgkZX5tcNe/gnGG3fvT3QXO2GNOXG1vsXfpmdMJMeZoJr/UWtpAVgC/s5PZw91Y/KJbVzpIVgC/s5IZD/e8Uk+yzZAXgC1vEmrOD98JW2wrAF1bYObIC8IUVdo6sAPx5tBenPpz9LXbV4UhhX7AVgD+H9q+uvI8f1SvfcPO/6uL4P1+5cuV38Zvhf/69K+9LfeirVxDxv/N7aAX2V8MKwJ9D+1dXrny1hID/qpu/C+C/ceV3v/G7V34X8H3ld//tR7+Hh756RQP+/Y9+B2yB4haWoxWAP4cGgL8ijwD/h/g4flGP2vVjWP5h5CarmFrSgL963Qzrr16/ahz/cCbP/+sj+Of9K+ALoP8f8Rvi/SsG8FcKsK+QFYA/h/avrvwODuo14OExfOWj6/AYhuPfuvKtkvzoin7+v/8+/PLDb8Af/xZAjAcRt78Lnz+8Cie9f+Wj6Ty/db2kAX8d8gCQ/26p+tE39JBeXvnG9W9Vl3+ThSVaAfhzaP/qyrd+B0CNgP+3V/7rt77x0ZWr37gCgP0vV0qljz76xvUfAmjfv/K71/8LQP/6R+8D4K/8jj74O/AJQ3YA/EdfTRjlX7/yP0CGOGD46If4twb8v71yxXyFFLYKVgD+HBoAvvTRFYmA/yFi8htXvvEiwBj//5aG5pUf4sMaAPxDnJM
2017-02-25 13:23:51 -05:00
"text/plain": [
2024-08-07 15:50:10 -04:00
"<pdfplumber.display.PageImage at 0x105cf6bd0>"
2017-02-25 13:23:51 -05:00
]
},
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
],
2016-02-20 19:42:37 -05:00
"source": [
2017-02-25 13:23:51 -05:00
"im.reset().debug_tablefinder()"
2016-02-20 19:42:37 -05:00
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
2017-02-25 13:23:51 -05:00
"The default settings correctly identify the table's vertical demarcations, but don't capture the horizontal demarcations between each group of five states/territories. So:"
]
},
{
"cell_type": "markdown",
2021-12-24 13:29:47 -05:00
"metadata": {},
2017-02-25 13:23:51 -05:00
"source": [
"### Using custom `.extract_table`'s settings\n",
2016-02-20 19:42:37 -05:00
"\n",
2017-02-25 13:23:51 -05:00
"- Because the columns are separated by lines, we use `vertical_strategy=\"lines\"`\n",
"- Because the rows are, primarily, separated by gutters between the text, we use `horizontal_strategy=\"text\"`\n",
2021-12-24 13:29:47 -05:00
"- To snap together a handful of the gutters at the top which aren't fully flush with one another, we use `snap_y_tolerance`, which snaps horizontal lines within a certain distance to the same vertical alignment.\n",
2017-02-25 13:23:51 -05:00
"- And because the left and right-hand extremities of the text aren't quite flush with the vertical lines, we use `\"intersection_tolerance\": 15`"
2016-02-20 19:42:37 -05:00
]
},
{
"cell_type": "code",
2017-02-25 13:23:51 -05:00
"execution_count": 6,
2022-05-31 15:50:50 -04:00
"metadata": {
"execution": {
2022-07-18 10:48:11 -04:00
"iopub.execute_input": "2022-07-18T14:47:51.177057Z",
"iopub.status.busy": "2022-07-18T14:47:51.176941Z",
"iopub.status.idle": "2022-07-18T14:47:51.178720Z",
"shell.execute_reply": "2022-07-18T14:47:51.178476Z"
2022-05-31 15:50:50 -04:00
}
},
2017-02-25 13:23:51 -05:00
"outputs": [],
"source": [
"table_settings = {\n",
" \"vertical_strategy\": \"lines\",\n",
" \"horizontal_strategy\": \"text\",\n",
2021-12-24 13:29:47 -05:00
" \"snap_y_tolerance\": 5,\n",
" \"intersection_x_tolerance\": 15,\n",
2017-02-25 13:23:51 -05:00
"}"
]
},
{
"cell_type": "code",
"execution_count": 7,
2022-05-31 15:50:50 -04:00
"metadata": {
"execution": {
2022-07-18 10:48:11 -04:00
"iopub.execute_input": "2022-07-18T14:47:51.180163Z",
"iopub.status.busy": "2022-07-18T14:47:51.180061Z",
"iopub.status.idle": "2022-07-18T14:47:51.339711Z",
"shell.execute_reply": "2022-07-18T14:47:51.339432Z"
2022-05-31 15:50:50 -04:00
}
},
2017-02-25 13:23:51 -05:00
"outputs": [
{
"data": {
2024-08-07 15:50:10 -04:00
"image/png": "iVBORw0KGgoAAAANSUhEUgAAA/AAAAJkCAMAAACBATrdAAADAFBMVEXMzP7+/v76BgnzKzajzP69vOcrK/7rKzY0CNNWVojIAAhmZpiVlcd3d6mmptiGhrg2AcpISHr+NjYrAMm3u/LHtuDMzDLM//+8weOuvvQ3N/9FVYh+frHMzNWtruB3lcdTZ5o2N2leXpBqhbdtbqCcw/UnKFpbc6U4R3kCAjTjHCaTuOpjeayEpdedntAAAACNjsBOToE9PnDqIixNYZPMATMXGUpyir30LAsqNWeNseN+ntAVFegAAMinteksLNP0LC2YvvCpmMcpLmCWljKKruBxjsBzaZiWibZYbqA+ToCFeai4p9dLXpAcJFa3aJp4dphnZzKIiDJWVXpjWolXVzLdFiFkfrBnZYioqDJ3dzKJlMqIhqi3tzJSS3vEKFq1dqiZpNvHF0mXlboZHlBISDKxiLoMDkC0t9u7VYdERGuoqLilpcsNEUO4uMWWlqppaWmdquDCNmjFtduRha4KBvq+RnjaEx3KDkBBO2tyco2Nkbezs7pYWFjDw8qzpcuFhZytntB3d3ehlLwdHvHAruCwjsBtcJmtstVdYIqGhoadocZ9gahWToA5OTKOm9F5bqBOUHzo6OgODuGzfrCqqqrW1tYICdvjGxsqKjKWlpYiIk0+QG0hHlBISEiej8AxLV4sN//AQHLDLmDAP3FgYH7HH1G9ToAUFDJEPnAgHk8jI++6XpBpX5CwkMIQD+GLfrA1NVwzL2EPEeS2bqAjIyMeIfQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACuro3mAAAACXBIWXMAAAsTAAALEwEAmpwYAADHVUlEQVR4nO39CWMbR5YmimaVEKJDsVW2IY/diQYSALHQWC5IUWST9FO3ZGtfSrZlS7Zlu7xU2e1y7TVdXXv1Nr3v0z23Z+bOzJ3l7ve+/f2+d05kAgSQSwTEJAyKcUwLZMbJyNgOMiK+OOfzvuTEiZNTI94XXQAnTpwsT5zBO3FyisQZvBMnp0icwTtxcorEGbwTJ6dInME7cXKKxBm8EyenSJzBO3FyisQZvBMnp0icwTtxcorEGbwTJ6dInME7cXKKxBm8EyenSJzBO3FyisQZvBMnp0icwTtxcorEGbwTJ6dInME7cXKKxBm8EyenSJzBO3FyisQZvBMnp0icwTtxcorEaPDPfOXZc06cODkJ8uwzv35Eg//1L537F//jv3DixMnqy//4L859yWDxBoOHu8/9C2fxTpycAAE7Pfclg8WbDf7La2vPrGn5shMnTlZXvvLlLxdh8KXS2ZKTVZBeL/nbCsrV1hddglMqZ0trRzV4uPvLzt6/ENnY0Ebd3oNfA/jlcrCzsfGtPvzWCjbgtzDWCzciKe3tLZT/rr7r5UEypb9xedHC9jb68W+1OuTahlw7G+lfSr2NzqK5O7GUs2tHXcODxX/F2fsXIhsb2oDbL0cG32pvdJud9katVAo2uv3O3sZ2pBduhB2U0vb2QvnvbtzqhMEnKWbZzzDVHJkY/MHGzvZu8MlOyxn8FyHPPHvUXXo9pXfyBQi8KB+WDg2+u9GEvy5/C0xpp43pOzuRXjhrWPv6n/hSb3B56moL/+2N59u7+N1RqkXmtx+/6Ft446HB6zsm91+dt+DW1fgxrbHBtza+1dN5B2jw4/TL8WcJPw8Nvne15KRYwSm9M/iTKRvdj9Hwxgb/yZa+WtttlT7eQaMa1CK9icHDlL6zsbfxcan5rY2Ne2DBu7AG2OiWoqudDVgI7AzaG3p9UDo0+CZOGaJvl97LcGNNGzyol1pduGPvXpxrfyPK9eW98bPwNrh7cG9jox4bvM4OpHkA6XD7xjWw8nb83QX639rWBj/45F6vB7dt3F1ac54OcQZ/cmUjOMBJfWzwVzfCSUoTZvvhZO0dbnSaIK3ICHdu9gcbe7X+zk6rtdGuHXTB5PRV+KfWAfuvdeMvCJzS3ww+ebkF+QWD5s5GqXVvp3Pw8k4LDB7tvXRrI6ztbdyL7r8GuXZ2vtXSmwrRsz5u1nY+KbV2dpr9ndjgw43JWxvuutbc+VYJvp/6kE2ttA2PCTZqYPBo72D+tcGtjdqymvN0iDP4kytgcgGYUWzw41enllo3ftuixJt2g8gIazj57+GXQrOHXwoDeJ/qq/qfnY0WXIlyijbtNrqtUm0b/tze6B3o93Uw6G+E+tW7sYVz9HvRrVuYax8LNDH4mp5e6JlCLTb47sZkh16nb2N6B7PplnZgslEKmr2N4Fv3YKXR/aRWau2uMtpwAsUZ/MkVnFPDpD42+P2pNzzKAcyPI9sKNwa9HsJ02ghxEbDx8ccf74B+L+zCtLkTXe1swDr85ZdxDT02eDDIXh/n/LVgD2b/vfHavR9P1bUR6yk9XNcbB2CsUwYPVx9u9EIsR2vyhp/MPDrR985+uLED5dm4N1679yB3XJ4c4LeNe8EXK87gT67gpBom9eM1/EZdX+3cOzjY00b1MJ4Oz67hwfjufUvv2h/sf7LRDZva4Fux/eH6e8bg4a4NMPCPg2aA8/ixwW/f+6QXTwW696L7P44MPowM/t60we8f7tKPJyLjLwRt8NtYnOahwb8caK1ef2tjet7i5OjiDP7kit5cg/f47C79xsblHr6U0Syjl2nC4Lu4jh50a/pte81k8B/vlNq43w8LgVr0Xm/Cm34AL2Gchpda34oNfi+a0u/iKx+SDg1e7/P3x7v039K79E24fWzwTZybtG51SrrYO100fPw+2Q6xLN1lNunTL87gT65og4dJ/RiHv7cRNMMdtK7uxl4H3sj3Ir2EwR9stJu1ezut2sbd/d1vgbllGXy30wnbG9uQX+1qiLsAH+90amD9OLXfBq1go37z443Y4Gsbe9c6OzuX4erNWn3K4Fv3dvrN8aYdqH0cNruY2djgWzvfujmAR8CNASxEBnqXHuYr4UZn0JlbqDg5ojiDP7nyiYbPDsYGX9q/BevsHW1X29+Kdtu0JAxew3J7evduYyPcia/q43PTBn9N79ndA4sb7MFnB0wWQbu9CJbD7YPS9s5Gt92Oc+1/ArMNnL2jUncvyrAJWvtwoTs5abeLJ+1w0RGn75cGL29sfHwzOiD4cSea/cOkHkG/SSWcFCPO4J8qmTpLv5+nNz5d08rXSsn48vSueQfXDDv1yd/7rflizD5uLImz9K34hrnytFKO9To5khRg8O5o7amVezvXBoHbVjtJcvSjtc555vQKTvW/5RbZJ0iO7jwz4w/vxImTFZZn1grxh3cBMJw4OQniAmA4cXKKxAXAOJ3ywsyHtb5BLllccfLFiguAcRrlX118Hz+qF79mp3/JRvF/vnjx4u/iN8P//HsX35f60lcvosX/zu+hONtfDXEBME6h/KuLF79aQoP/qp2+jcF/7eLvfu13L/4u2PfF3/3XH/4eXvrqRW3w73/4OyBHKK6TAsUdvDmFAgZ/UU4M/vfxdfyCnrXr17D8/VhNVjG1pA3+0pVoWn/pyqVI8ffn8vyfPoR/3r8IumD0/xd+Q7x/MTL4i87YV0icwZ9C+VcXfwcn9drg4TV88cMr8BqG69+4+I2S/PCifv+//z788v2vwR//GowYL6Ld/i58fv8S3PT+xQ9n8/zGlZI2+CuQBxj575aqH35NT+nlxa9d+UZ1+ZV0kirO4E+h/KuL3/gdMGo0+H998b9842sfXrz0tYtgsP/5Yqn04Ydfu/J9MNr3L/7ulf8Mpn/lw/fB4C/+jr74O/AJU3Yw+A+/mjLLv3Lxv0KGOGH48Pv4tzb4f33xYvQV4mQ
2017-02-25 13:23:51 -05:00
"text/plain": [
2024-08-07 15:50:10 -04:00
"<pdfplumber.display.PageImage at 0x105cf6bd0>"
2017-02-25 13:23:51 -05:00
]
},
"execution_count": 7,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"im.reset().debug_tablefinder(table_settings)"
]
},
{
"cell_type": "code",
"execution_count": 8,
2022-05-31 15:50:50 -04:00
"metadata": {
"execution": {
2022-07-18 10:48:11 -04:00
"iopub.execute_input": "2022-07-18T14:47:51.341430Z",
"iopub.status.busy": "2022-07-18T14:47:51.341319Z",
"iopub.status.idle": "2022-07-18T14:47:51.527088Z",
"shell.execute_reply": "2022-07-18T14:47:51.526735Z"
2022-05-31 15:50:50 -04:00
}
},
2017-02-25 13:23:51 -05:00
"outputs": [],
"source": [
"table = p0.extract_table(table_settings)"
]
},
{
"cell_type": "code",
"execution_count": 9,
2022-05-31 15:50:50 -04:00
"metadata": {
"execution": {
2022-07-18 10:48:11 -04:00
"iopub.execute_input": "2022-07-18T14:47:51.528833Z",
"iopub.status.busy": "2022-07-18T14:47:51.528741Z",
"iopub.status.idle": "2022-07-18T14:47:51.530609Z",
"shell.execute_reply": "2022-07-18T14:47:51.530374Z"
2022-05-31 15:50:50 -04:00
}
},
2016-02-20 19:42:37 -05:00
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
2021-12-24 13:29:47 -05:00
"['NICS Firearm Background Checks', None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None]\n",
2017-02-25 13:23:51 -05:00
"['November - 2015', None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None]\n",
2021-12-24 13:29:47 -05:00
"['', '', None, None, None, None, None, 'Pre-Pawn', None, None, 'Redemption', None, None, 'Returned/Disposition', None, None, 'Rentals', None, 'Private Sale', None, None, 'Return to Seller - Private Sale', None, None, '']\n",
2017-02-25 13:23:51 -05:00
"['State / Territory', 'Permit', 'Handgun', 'Long Gun', '*Other', '**Multiple', 'Admin', 'Handgun', 'Long Gun', '*Other', 'Handgun', 'Long Gun', '*Other', 'Handgun', 'Long Gun', '*Other', 'Handgun', 'Long Gun', 'Handgun', 'Long Gun', '*Other', 'Handgun', 'Long Gun', '*Other', 'Totals']\n",
2021-12-24 13:29:47 -05:00
"['Alabama', '18,870', '23,022', '22,650', '859', '1,178', '0', '14', '15', '0', '2,179', '2,307', '11', '0', '0', '0', '', '', '13', '14', '0', '3', '2', '0', '71,137']\n"
2016-02-20 19:42:37 -05:00
]
}
],
"source": [
2016-03-06 18:30:41 -05:00
"for row in table[:5]:\n",
" print(row)"
2016-02-20 19:42:37 -05:00
]
},
{
"cell_type": "markdown",
2021-12-24 13:29:47 -05:00
"metadata": {},
2016-02-20 19:42:37 -05:00
"source": [
2017-02-25 13:23:51 -05:00
"### Cleaning up the data\n",
2016-02-20 19:42:37 -05:00
"\n",
2017-02-25 13:23:51 -05:00
"`.extract_table` worked with our custom settings, but the table it detected contains extraneous headers and footers. Since we know that the `Alabama` row is the first, and that there are 56 rows we care about (50 states + DC + 4 territories + the \"Totals\" row), we can slice away the rest:"
2016-02-20 19:42:37 -05:00
]
},
{
"cell_type": "code",
2017-02-25 13:23:51 -05:00
"execution_count": 10,
2022-05-31 15:50:50 -04:00
"metadata": {
"execution": {
2022-07-18 10:48:11 -04:00
"iopub.execute_input": "2022-07-18T14:47:51.532049Z",
"iopub.status.busy": "2022-07-18T14:47:51.531967Z",
"iopub.status.idle": "2022-07-18T14:47:51.533676Z",
"shell.execute_reply": "2022-07-18T14:47:51.533426Z"
2022-05-31 15:50:50 -04:00
}
},
2016-02-20 19:42:37 -05:00
"outputs": [],
2017-02-25 13:23:51 -05:00
"source": [
2021-12-24 13:29:47 -05:00
"core_table = table[4:4+56]"
2017-02-25 13:23:51 -05:00
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The first row:"
]
},
{
"cell_type": "code",
"execution_count": 11,
2022-05-31 15:50:50 -04:00
"metadata": {
"execution": {
2022-07-18 10:48:11 -04:00
"iopub.execute_input": "2022-07-18T14:47:51.535244Z",
"iopub.status.busy": "2022-07-18T14:47:51.535160Z",
"iopub.status.idle": "2022-07-18T14:47:51.537352Z",
"shell.execute_reply": "2022-07-18T14:47:51.537122Z"
2022-05-31 15:50:50 -04:00
}
},
2017-02-25 13:23:51 -05:00
"outputs": [
{
"data": {
"text/plain": [
"'Alabama • 18,870 • 23,022 • 22,650 • 859 • 1,178 • 0 • 14 • 15 • 0 • 2,179 • 2,307 • 11 • 0 • 0 • 0 • • • 13 • 14 • 0 • 3 • 2 • 0 • 71,137'"
]
},
"execution_count": 11,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"\" • \".join(core_table[0])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The last:"
]
},
{
"cell_type": "code",
"execution_count": 12,
2022-05-31 15:50:50 -04:00
"metadata": {
"execution": {
2022-07-18 10:48:11 -04:00
"iopub.execute_input": "2022-07-18T14:47:51.538797Z",
"iopub.status.busy": "2022-07-18T14:47:51.538719Z",
"iopub.status.idle": "2022-07-18T14:47:51.540722Z",
"shell.execute_reply": "2022-07-18T14:47:51.540498Z"
2022-05-31 15:50:50 -04:00
}
},
2017-02-25 13:23:51 -05:00
"outputs": [
{
"data": {
"text/plain": [
"'Totals • 804,006 • 671,330 • 636,903 • 26,597 • 23,015 • 1,281 • 218 • 249 • 13 • 29,905 • 38,487 • 102 • 1,656 • 533 • 44 • 0 • 0 • 1,067 • 905 • 65 • 31 • 45 • 5 • 2,236,457'"
]
},
"execution_count": 12,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"\" • \".join(core_table[-1])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now, let's turn those rows into dictionaries, and also convert strings-representing-numbers to the numbers themselves, e.g., `\"18,870\" -> 18870`:"
]
},
{
"cell_type": "code",
"execution_count": 13,
2022-05-31 15:50:50 -04:00
"metadata": {
"execution": {
2022-07-18 10:48:11 -04:00
"iopub.execute_input": "2022-07-18T14:47:51.542220Z",
"iopub.status.busy": "2022-07-18T14:47:51.542120Z",
"iopub.status.idle": "2022-07-18T14:47:51.544117Z",
"shell.execute_reply": "2022-07-18T14:47:51.543851Z"
2022-05-31 15:50:50 -04:00
}
},
2017-02-25 13:23:51 -05:00
"outputs": [],
2016-02-20 19:42:37 -05:00
"source": [
"COLUMNS = [\n",
" \"state\",\n",
" \"permit\",\n",
" \"handgun\",\n",
" \"long_gun\",\n",
" \"other\",\n",
" \"multiple\",\n",
" \"admin\",\n",
" \"prepawn_handgun\",\n",
" \"prepawn_long_gun\",\n",
" \"prepawn_other\",\n",
" \"redemption_handgun\",\n",
" \"redemption_long_gun\",\n",
" \"redemption_other\",\n",
" \"returned_handgun\",\n",
" \"returned_long_gun\",\n",
" \"returned_other\",\n",
" \"rentals_handgun\",\n",
" \"rentals_long_gun\",\n",
" \"private_sale_handgun\",\n",
" \"private_sale_long_gun\",\n",
" \"private_sale_other\",\n",
" \"return_to_seller_handgun\",\n",
" \"return_to_seller_long_gun\",\n",
" \"return_to_seller_other\",\n",
" \"totals\"\n",
"]"
]
},
{
"cell_type": "code",
2017-02-25 13:23:51 -05:00
"execution_count": 14,
2022-05-31 15:50:50 -04:00
"metadata": {
"execution": {
2022-07-18 10:48:11 -04:00
"iopub.execute_input": "2022-07-18T14:47:51.545556Z",
"iopub.status.busy": "2022-07-18T14:47:51.545454Z",
"iopub.status.idle": "2022-07-18T14:47:51.547305Z",
"shell.execute_reply": "2022-07-18T14:47:51.547053Z"
2022-05-31 15:50:50 -04:00
}
},
2016-02-20 19:42:37 -05:00
"outputs": [],
"source": [
2016-03-06 18:30:41 -05:00
"def parse_value(i, x):\n",
" if i == 0: return x\n",
2017-02-25 13:23:51 -05:00
" if x == \"\": return None\n",
2016-02-20 19:42:37 -05:00
" return int(x.replace(\",\", \"\"))"
]
},
{
"cell_type": "code",
2017-02-25 13:23:51 -05:00
"execution_count": 15,
2022-05-31 15:50:50 -04:00
"metadata": {
"execution": {
2022-07-18 10:48:11 -04:00
"iopub.execute_input": "2022-07-18T14:47:51.548636Z",
"iopub.status.busy": "2022-07-18T14:47:51.548542Z",
"iopub.status.idle": "2022-07-18T14:47:51.550310Z",
"shell.execute_reply": "2022-07-18T14:47:51.550064Z"
2022-05-31 15:50:50 -04:00
}
},
2016-02-20 19:42:37 -05:00
"outputs": [],
"source": [
2017-02-25 13:23:51 -05:00
"from collections import OrderedDict\n",
2016-02-20 19:42:37 -05:00
"def parse_row(row):\n",
2021-12-24 13:29:47 -05:00
" return {COLUMNS[i]:parse_value(i, cell)\n",
" for i, cell in enumerate(row)}"
2016-02-20 19:42:37 -05:00
]
},
{
"cell_type": "code",
2017-02-25 13:23:51 -05:00
"execution_count": 16,
2022-05-31 15:50:50 -04:00
"metadata": {
"execution": {
2022-07-18 10:48:11 -04:00
"iopub.execute_input": "2022-07-18T14:47:51.551772Z",
"iopub.status.busy": "2022-07-18T14:47:51.551667Z",
"iopub.status.idle": "2022-07-18T14:47:51.553680Z",
"shell.execute_reply": "2022-07-18T14:47:51.553445Z"
2022-05-31 15:50:50 -04:00
}
},
2016-02-20 19:42:37 -05:00
"outputs": [],
"source": [
2017-02-25 13:23:51 -05:00
"data = [ parse_row(row) for row in core_table ]"
2016-02-20 19:42:37 -05:00
]
},
{
"cell_type": "markdown",
2021-12-24 13:29:47 -05:00
"metadata": {},
2016-02-20 19:42:37 -05:00
"source": [
2017-02-25 13:23:51 -05:00
"Now here's the first row, parsed:"
2016-02-20 19:42:37 -05:00
]
},
{
"cell_type": "code",
2017-02-25 13:23:51 -05:00
"execution_count": 17,
2022-05-31 15:50:50 -04:00
"metadata": {
"execution": {
2022-07-18 10:48:11 -04:00
"iopub.execute_input": "2022-07-18T14:47:51.555189Z",
"iopub.status.busy": "2022-07-18T14:47:51.555095Z",
"iopub.status.idle": "2022-07-18T14:47:51.557309Z",
"shell.execute_reply": "2022-07-18T14:47:51.557061Z"
2022-05-31 15:50:50 -04:00
}
},
2016-02-20 19:42:37 -05:00
"outputs": [
{
"data": {
"text/plain": [
2021-12-24 13:29:47 -05:00
"{'state': 'Alabama',\n",
" 'permit': 18870,\n",
" 'handgun': 23022,\n",
" 'long_gun': 22650,\n",
" 'other': 859,\n",
" 'multiple': 1178,\n",
" 'admin': 0,\n",
" 'prepawn_handgun': 14,\n",
" 'prepawn_long_gun': 15,\n",
" 'prepawn_other': 0,\n",
" 'redemption_handgun': 2179,\n",
" 'redemption_long_gun': 2307,\n",
" 'redemption_other': 11,\n",
" 'returned_handgun': 0,\n",
" 'returned_long_gun': 0,\n",
" 'returned_other': 0,\n",
" 'rentals_handgun': None,\n",
" 'rentals_long_gun': None,\n",
" 'private_sale_handgun': 13,\n",
" 'private_sale_long_gun': 14,\n",
" 'private_sale_other': 0,\n",
" 'return_to_seller_handgun': 3,\n",
" 'return_to_seller_long_gun': 2,\n",
" 'return_to_seller_other': 0,\n",
" 'totals': 71137}"
2016-02-20 19:42:37 -05:00
]
},
2017-02-25 13:23:51 -05:00
"execution_count": 17,
2016-02-20 19:42:37 -05:00
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
2017-02-25 13:23:51 -05:00
"data[0]"
2016-02-20 19:42:37 -05:00
]
},
{
"cell_type": "markdown",
2021-12-24 13:29:47 -05:00
"metadata": {},
2016-02-20 19:42:37 -05:00
"source": [
"### Sort the data\n",
"\n",
"For demonstration purposes, let's list the rows with the highest number of handgun-only background checks:"
]
},
{
"cell_type": "code",
2017-02-25 13:23:51 -05:00
"execution_count": 18,
2022-05-31 15:50:50 -04:00
"metadata": {
"execution": {
2022-07-18 10:48:11 -04:00
"iopub.execute_input": "2022-07-18T14:47:51.558770Z",
"iopub.status.busy": "2022-07-18T14:47:51.558674Z",
"iopub.status.idle": "2022-07-18T14:47:51.560629Z",
"shell.execute_reply": "2022-07-18T14:47:51.560371Z"
2022-05-31 15:50:50 -04:00
}
},
2016-02-20 19:42:37 -05:00
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Totals: 671,330 handgun-only checks\n",
"Pennsylvania: 62,752 handgun-only checks\n",
"Texas: 56,941 handgun-only checks\n",
"Florida: 50,796 handgun-only checks\n",
"California: 41,181 handgun-only checks\n",
"Ohio: 34,878 handgun-only checks\n"
]
}
],
"source": [
2017-02-25 13:23:51 -05:00
"for row in list(reversed(sorted(data, key=lambda x: x[\"handgun\"])))[:6]:\n",
2016-02-20 19:42:37 -05:00
" print(\"{state}: {handgun:,d} handgun-only checks\".format(**row))"
]
},
{
"cell_type": "markdown",
2021-12-24 13:29:47 -05:00
"metadata": {},
2016-02-20 19:42:37 -05:00
"source": [
2016-03-06 20:21:54 -05:00
"### Use `extract_text` to extract the report month\n",
2016-02-20 19:42:37 -05:00
"\n",
2017-02-25 13:23:51 -05:00
"It looks like the month of the report is listed in an area 35px to 65px from the top of the page. But there's also some other text directly above and below it. So when we crop for that area, we'll use `.within_bbox` instead of `.crop` to select only characters (and other objects) that are fully within the bounding box."
2016-02-20 19:42:37 -05:00
]
},
{
"cell_type": "code",
2017-02-25 13:23:51 -05:00
"execution_count": 19,
2022-05-31 15:50:50 -04:00
"metadata": {
"execution": {
2022-07-18 10:48:11 -04:00
"iopub.execute_input": "2022-07-18T14:47:51.562125Z",
"iopub.status.busy": "2022-07-18T14:47:51.562027Z",
"iopub.status.idle": "2022-07-18T14:47:51.728407Z",
"shell.execute_reply": "2022-07-18T14:47:51.728150Z"
2022-05-31 15:50:50 -04:00
}
},
2017-02-25 13:23:51 -05:00
"outputs": [
{
"data": {
2023-07-16 17:33:25 -04:00
"image/png": "iVBORw0KGgoAAAANSUhEUgAAA/AAAAAeCAMAAAC8JwVTAAADAFBMVEXL/v7///8AAADb6fH//wD96dnr8d7y3Nv+/sv+AQG56Oh3lpbjhYXT2dnQ5+f6FhbzNjY3RUXatLTglJSEpqb2JibmeHgrNzfpaGjcqKjWxsVGWFgcIyNxjo5qhYXO7O3tV1cTGBjwRUWcxMRUZ2iw3d1ie3siKysxPDySt7eiy8sMEBBcc3PfnZ2r19emmJSNsbFXWFdNYWFFRUXvTk5FRD1FPj7q1NNSUUeDd3bBytOippnhzcy2xsuJj5ZjWlpyclt0d21xfH7EsrHBsqXR1sWThYXVxLZpaVSBjI+eopXb4M+mtrni0MJSS0qEkJPD1dqQnaHJyaB3bGybm3y+w7Ti6NYAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACUm2nUAAAACXBIWXMAAAsTAAALEwEAmpwYAAAF1ElEQVR4nO2aCZfaNhCAVaVNOxif2AZsY+4b9t5N0jRN7/u+/v9P6YxkbrpLNoRl6/neW2RL4zHse580Moj37kIK5iEYXwjRlm65LITs4J+remv9dtsd0ZEhh6qnJwv6gsFA1Oi43zSIdnc8jlNfGrrXoJcyJilgD+Hr612JQ0nHn8hCLctkyF55XBAj6eNJ3NfXJ/g+8NqeykEvRpY1ld1F0qG6BrNm477spnJCb8cvyJoaKsjyREUVDFdmlzH7Qt6ls2DhjxQSXkykzISPtSRjWSjImMYNqbzfFD6WJ0KM4nYq20KcoWa3Cp80RTkRdFVB2dqWPoaOaH5R92lmwrs6yZkYkPlyIfyQ8hkzc5vNgsq9EN6XKfZc1HS6JKb793E+6fXovcSH+nfmBBb+0aKEF8lMeNGXHb+X0DIZS9fAFbmv4zaEb8uyP+w30fVJ96yJuv2X8Be1NC3jmh3L4UkqcZpImsYQ7afQHso/kW6ayEz4oRycGaRzR6ZDd0l4qij8ZCZ8WyapH1OymfCYNR3hLTBdp92Rbbo/zSepNEY1NRkw+4OFf7SMyXKyNxO+e5FI2VReTZpSztfGufDuQEsmfBx22zQxSJkmS2pmwuuq+0wSfVxnTwbY1tDDk7K6UIUmtAgnMsYthc5qjLHaoOqdguL5Cl4QXeyI57W5j3W6HIzEQnjKmpDYHWxresKhoj5e+hDMnmDh/08UCvOj7u2BWXt71JbEizsgNdozJO78vLstaEvHya7vZzOQeUtYeObe9Jtnow4/VntUsPDMvWljdd7kTfajgoVn3ob14p05clh4hskRLDzD5AgWPofUV5qd4+/A2gi3tgcyDwgLnz88cKgxobhbvAmlu4MqANAg1Ss2BKbqCsFS5wS7fySw8PnDA6X6Gwh/d2AJGqUGNEj8xtS2qasISvjAriD3f7fMXmHh8wcKD+bM43pEy3FdleFqGTajLEwN0AkGWpGu063I0oHRWs4goBcQFknvYUlQd0ALDyz7McHC5w8PKlTUK+FD1NL2hENrchWqIgpArf9OYAO0Sjg4xUDqDHFaaGDbsnDtdsBezVmlGQCF98DDA7TetEtFEj6Cklc1D/8hme2w8PnDg2oFpSbhp+BUizZYJfL0CoSw7ZLXQvEdaHhXqL5nOxgIFdVZgUqEJTsKbxe3bOs9uMKEZL7donMl/BRATyHMUcDC5w8UHhdjk4R3yMkSFOuocR2L76pSE1o4QItzi0oAFN4RSmLV4rRQxASbRGDT4k+reUCBWvgQQqsawPoWgHkgWPj8QcJ74JDwNm29TTS9Aeh9RBW+bdsQYEmf1fxorUnlPFb9ph6ESFuNhEEQzLboVbAjoZKsrPCKEi/xxwILnz9IeHqavhA+pD5yPIRKCZkKx1kWnnS1A1zpabBkzU0uOo4T6sOp/jIu28OrWUCFWerRH4SH/5jMNlj4/KGEx6IePW7RWq0qdNshravKzKtwTfgGnTQE0PQQXi2EX1AFRz/jJ9c9/cW9ClN3CHf5Jp85BCx8/tDCq2/jPQhKIe29qZgnY20IvRau0qvCQ1FtwyvQqobqod2G8DZUQkTg3qDkBfoRfvbQzsFr7PV45oFg4fOHFl7Qk3rcXQPu5gXZrfbdpgMQ0NdyTvYLuyLUcQijpuoSsBvWFuE90GAJjwkc/YhOhxVnd2COARY+76y7WN/6K1hzrb0Fa/2n9+bOP9pn3jksPMPkCBaeYXIEC88wOeLohJcMw9yHRyr8+48Q+eTQyA8OzAN8xI/2h3x6aOSHh4aFPxws/Lv4iCw8C3+kvIkNH7uXn24evrENu4p6ef38Z3Vwvtz77Venp3sU/nv38ndsXv202l3vPOl03PPfzq2V7i9e/Wqd7/ARbzP4s+vnf2SHf9HLd/sS/vL686dP/znXJz/+oJqvn1//jc3Ll28jfHRz8/oZtt8s9X1yqtsXgxf4+uz16enS4OXNn9h3vhIZ/XLzJTYrcXsW/l+JJaqgg1lBngAAAABJRU5ErkJggg==\n",
2017-02-25 13:23:51 -05:00
"text/plain": [
2024-08-07 15:50:10 -04:00
"<pdfplumber.display.PageImage at 0x106a70080>"
2017-02-25 13:23:51 -05:00
]
},
"execution_count": 19,
"metadata": {},
"output_type": "execute_result"
}
],
2016-02-20 19:42:37 -05:00
"source": [
2017-02-25 13:23:51 -05:00
"month_crop = p0.within_bbox((0, 35, p0.width, 65))\n",
"month_crop.to_image()"
2016-02-20 19:42:37 -05:00
]
},
{
"cell_type": "code",
2017-02-25 13:23:51 -05:00
"execution_count": 20,
2022-05-31 15:50:50 -04:00
"metadata": {
"execution": {
2022-07-18 10:48:11 -04:00
"iopub.execute_input": "2022-07-18T14:47:51.729999Z",
"iopub.status.busy": "2022-07-18T14:47:51.729896Z",
"iopub.status.idle": "2022-07-18T14:47:51.735619Z",
"shell.execute_reply": "2022-07-18T14:47:51.735389Z"
2022-05-31 15:50:50 -04:00
}
},
2016-02-20 19:42:37 -05:00
"outputs": [
{
"data": {
"text/plain": [
"'November - 2015'"
]
},
2017-02-25 13:23:51 -05:00
"execution_count": 20,
2016-02-20 19:42:37 -05:00
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
2017-02-25 13:23:51 -05:00
"month_chars = month_crop.extract_text()\n",
2016-03-06 18:30:41 -05:00
"month_chars"
2016-02-20 19:42:37 -05:00
]
},
{
"cell_type": "markdown",
2021-12-24 13:29:47 -05:00
"metadata": {},
2016-02-20 19:42:37 -05:00
"source": [
"---\n",
"\n",
"---\n",
"\n",
"---"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
2024-08-07 15:50:10 -04:00
"version": "3.12.4"
2016-02-20 19:42:37 -05:00
}
},
"nbformat": 4,
2021-12-24 13:29:47 -05:00
"nbformat_minor": 1
2016-02-20 19:42:37 -05:00
}