SU/sets spreadsheets and parsing scripts

Discuss Median XL!
Milkshake
Abomination
142 | 17
Common Popularity Badge
Has a thread with over 10.000 views
Hi all. I wanted to mess around and do a holy grail, so I created a parsing script that outputs all SUs and sets to CSV files.

You can find the current (MXL v1.8) spreadsheets here:
SUs
Sets

You can also use the below PowerShell scripts to recreate the CSVs (without formatting). These scripts may only work with the current documentation, but could likely be updated easily. Please don't judge my scripts too much; I got a bit tired of trying to parse and got sloppy xD

SUs PowerShell script

Code: Select all

$suurl = "https://docs.median-xl.com/doc/items/sacreduniques"
$surequest = Invoke-WebRequest $suurl
$tables = $surequest.ParsedHtml.getElementsByTagName("table") | Where-Object{$_.classname -like "uniques"}

$wholeList = [Ordered] @{}
$count=0

foreach ($table in $tables) {
    $checkRow = $table.rows[0].children | Where-Object{$_.tagname -like 'th'}
    if ($checkRow) {
        $itemClass = $checkRow.outerText.Trim()
        $wholeList.$itemClass = @()
        for ($n=1; $n -lt $table.rows.length; $n++) {
            for ($i=1; $i -lt $table.rows[$n].cells.length; $i++) {
                $itemName = ($table.rows[$n].cells[$i].outerText -split '\n')[0]
                $wholeList.$itemClass += $itemName.Trim()
            }
        }
    }
    else {
        $itemClass = "Jewelry$count"
        $count += 1
        $wholeList.$itemClass = @()
        for ($n=0; $n -lt $table.rows.length; $n++) {
            for ($i=1; $i -lt $table.rows[$n].cells.length; $i++) {
                $itemName = ($table.rows[$n].cells[$i].outerText -split '\n')[0]
                $wholeList.$itemClass += $itemName.Trim()
            }
        }
    }
}

$wholeList.GetEnumerator() |
    Select-Object -Property @{N='Item Class';E={$_.Key}},
    @{N='Item Name';E={$_.Value -join ', '}} |
    Export-Csv -NoTypeInformation -Path .\mxlsus.csv


Sets PowerShell script

Code: Select all

$setsurl = "https://docs.median-xl.com/doc/items/sets"
$setrequest = Invoke-WebRequest $setsurl
$tables = $setrequest.ParsedHtml.getElementsByTagName("table") | Where-Object{$_.classname -like "sets"}

$wholeList = [Ordered] @{}

foreach ($table in $tables) {
    $split = $table.rows[0].cells[0].outerText -split '\n'
    $set = $split[0]
    $wholeList.$set = @()
    $i = 2
    while ($split[$i].Trim() -ne "") {
        $wholeList.$set += $split[$i].Trim()
        $i++
    }
}

$wholeList.GetEnumerator() |
    Select-Object -Property @{N='Set Name';E={$_.Key}},
    @{N='Item Name';E={$_.Value -join ', '}} |
    Export-Csv -NoTypeInformation -Path .\mxlsets.csv


Feel free to use as you'd like. Let me know if there are any issues! (:
Edited by Milkshake 3 years.
Milkshake
Abomination
142 | 17
Common Popularity Badge
Has a thread with over 10.000 views
Updated with sets spreadsheet and script (:
boozek
Son of Lucion
733 | 31
Common Posting Badge
Posted over 1.000 messages
Common Popularity Badge
Has a thread with over 10.000 views
Common Love Badge
Earned over 20 cookies
Common Supporter Badge
Donated 1 time
might be dumb question or I just didnt figure it out can you tell me what can I use it for, give an example please
User avatar
Quirinus
Team Member
1510 | 184
Common Posting Badge
Posted over 1.000 messages
Legendary Popularity Badge
Has a thread with over 250.000 views
Great Love Badge
Earned over 100 cookies
Common Supporter Badge
Donated 1 time
Legendary Contribution Badge
Median XL Team Member
Nice to see a fellow coder! :)
"but if it's simple it's not that hard" - Quirinus 2017
"small things like this are not a big deal" - Quirinus 2017
Milkshake
Abomination
142 | 17
Common Popularity Badge
Has a thread with over 10.000 views
Quirinus wrote:Nice to see a fellow coder! :)

haha idk if I'd say I'm a coder. I'm mostly a sys engineer with a little programming background. Always down to help with any server deployment stuff (;

boozek wrote:might be dumb question or I just didnt figure it out can you tell me what can I use it for, give an example please


Uhh not sure what all you may want to use it for. Could be used for just about whatever related to MXL SUs and Sets. I'll be using it for a holy grail attempt xD