[WoW Armory] Looking for a PHP4 XML/XSL Guru

[Xlorep DarkHelm]

Ok. Here's the situation -- the Armory is filled with a lot of useful information. All of this information appears to be in an XML document, which then in turn is rendered into an "XHTML Transitional" document through the use of an XSL stylesheet. My problem is, I'd like to get the XML document before the transformation happens. This is mainly to expand a sidebar component of my personal website (DarkHelm.org (http://darkhelm.org), down the right side of the page) to be able to retrieve data directly from the Armory XML documents for my characters, to reflect character level increases automatically -- not much more than that, really, just something I thought would be cool to have automated, rather than me having to go in and update my internal data structure to reflect the new levels of my characters.

So, here is my problem -- whenever I retrieve the XML documents for my characters specifically, they come already XSL transformed. However, when I "view source" in either IE or FireFox, I can see the XML documents underlying the page, rather than the results of the transformation into XHTML -- which tells me it is possible to get the XML document without transforming it. But how?

My website is hosted by a company that only has PHP 4.4.0 on it, so a PHP 5.0 solution isn't going to work. It does have Expat, version 1.95 installed in it, so I have some rudimentary XML capabilities (Here's my phpinfo (http://darkhelm.org/php-tester.php) if interested). I have recently lost shell access to the server -- a policy change on their part which has infuriated me, however I don't have the resources currently to move to a different web host at this time.

I know there has to be a way to parse/process the XML document natively without all of the XSL transformation garbage. It would be a simple matter, once I can actually read the XML document into my code, to be able to parse out the information I seek.

Any help would be appreciated.

[Xlorep DarkHelm]

Well, I answered my own question, and found the solution. The sidebar of my personal website (http://darkhelm.org) also lists my WoW characters. The data on that sidebar is loaded directly from the Armory, so it is now automatically updated, and I could possibly expand it in the future to include other information.

Basically, the problem was the "user_agent" needed to be set to a modern browser's user_agent string, faking outtheir website to send the XML (as it does for the "newer style" browsers that can handle it) rather than the XHTML Transitional code that it otherwise will send.

[Xlorep DarkHelm]

Now that I've "cracked the code" so to speak, I've been having fun working on this. Right now, I've added the following features:


Fault tolerance -- the sidebar is safeguarded against Blizzard's site going down, being unresponsive, or otherwise being irritating. This is done in two different ways:
XML data frm the Armory is parsed, processed, and stored into a table locally in my website's database. If the XML files are unreachable, it simply skips processing the XML. The data used to show on the site is pulled from this table.
Images that are from Blizzard's site -- namely "default" character icons (when I don't specify an icon of my own), plus the race & class icons are copied from Blizzard's site to my own if I don't already have a copy, and then I use the local ones to show on my site.
Image defaults -- I might not get around to making a wowmodelviewer animated gif for my characters when I add them. As such, the system grabs the appropriate image from Blizzard's site based on race, class, and level, then uses that image instead.
Automatic ordering of characters: Based on realms, factions, guilds, and then characters. The sorting is made in what I think is an "intelligent" order. Groups with a higher amount of total XP are placed above those with lower total XP. On ties, the group with the highest level character wins. If tied again, the one with the largest number of characters wins. If even that is tied, then it is done alphabetically.
Simplistic way to add characters to the sidebar. I just have a two-dimensional array that contains "Realm" and "Name" values for each character. I add, update, or delete entries from this simple array to change who is displayed in the sidebar.
I've made it rather flexible in design. I am considering making it into something I can provide for people to use on their own sites that have PHP & MySQL.

[yertle82]

Can you please explain this:

"Basically, the problem was the "user_agent" needed to be set to a modern browser's user_agent string, faking outtheir website to send the XML (as it does for the "newer style" browsers that can handle it) rather than the XHTML Transitional code that it otherwise will send."

I realise I can do all the stuff you say in your last post. I just can't get the xml file. I am somewhat new to php. oh please help, I've looked at this for far to long :(

[yertle82]

YAH!!!! I got it to work. For anyone else has stumbles upon this, add:

ini_set('user_agent', '[a modern user agent's string]')
i.e. ini_set("user_agent", "Firefox/2.0.0.8");
to your source code or your php.ini file.

This site was quite helpful too:
http://altitis.blogspot.com/2007/09/coding-armory-crawler-in-php-basic.html

[Xlorep DarkHelm]

Sorry, never really noticed your comment. Glad you got it working.

[glock]

Hi there!

Im not so good at php and know nothing about XML, but I would love to learn how I can get XML data to my php website.

I have a wow-guild-site where I would like to show my guildmates and what there level is.

Can you in some way show me how I can do this ?

I know most of you hate to show your hard worked code just like that, but I really want to learn how its done, not just view the code.

So code with some comments and reasons would be very nice.

I hope you will reply on this for me, even in private :)

[Xlorep DarkHelm]

heh, I broke it down, in detail, on the previous incarnation of my website, which unfortunately I lost that old database. I guess I can do another write-up for the new site, especially as I've made a few small changes to it all.

[d.vel.oper]

I could have written it in like ~30 lines of Perl code.

lulz

[Xlorep DarkHelm]

I could have written it in like ~30 lines of Perl code.

lulz

true, but perl is fugly.

[steelcobra]

true, but perl is fugly.

One bash.org quote states that it's the code you see in the Matrix. Another says it's the only language where you can write an entire program in emoticons.

[d.vel.oper]

true, but perl is fugly.

Heh, and Perl's syntax is so different from PHP's how...?

[KN]

Heh, and Perl's syntax is so different from PHP's how...?

The actual PHP intepreter is written in perl, isn't it?

[Xlorep DarkHelm]

Heh, and Perl's syntax is so different from PHP's how...?

I didn't say that PHP was any better. It's pretty fugly too. BUT, PHP is what I'm forced to use if I want to build web apps that run, y'know, on most web hosts. Most web hosts find open access to Perl and the like to be a bit.... unsafe.

I still like Python, small, fast, and unlike the other two, it is precompiled into a fast bytecode. Heck, like I said, there's been significant efforts which have made Python be able to run *faster* and more efficiently than what C/C++ compilers can do. It also has a very readable and intuitive syntax. I can whip together things in Python without needing to look one thing up, things just work the way I think they should work.

The actual PHP intepreter is written in perl, isn't it?

No, in C or C++. There are Perl interpreters though.

[d.vel.oper]

The actual PHP intepreter is written in perl, isn't it?

The original version of PHP (wasn't called PHP at the time, if I recall) was a relatively simple Perl script, yes.

I didn't say that PHP was any better. It's pretty fugly too. BUT, PHP is what I'm forced to use if I want to build web apps that run, y'know, on most web hosts. Most web hosts find open access to Perl and the like to be a bit.... unsafe.

I still like Python, small, fast, and unlike the other two, it is precompiled into a fast bytecode. Heck, like I said, there's been significant efforts which have made Python be able to run *faster* and more efficiently than what C/C++ compilers can do. It also has a very readable and intuitive syntax. I can whip together things in Python without needing to look one thing up, things just work the way I think they should work.



No, in C or C++. There are Perl interpreters though.

lol, Python is decent, yes, now stop obsessing over it. :P

You're sounding like me when I talk about C++. As far as Python being faster than C++...heh, maybe in certain situations, but anything you can do to optimize Python, can be done almost tenfold in C++. Bytecode or not, it's still interpreted. I believe Python might be able to emit C/C++ code, like Perl can, but there will always be an extra VM/interpreter layer, then the C runtime layer, etc. If Python could emit object code and use a linker, then yeah, you're talking a real performance gain.

[Xlorep DarkHelm]

lol, Python is decent, yes, now stop obsessing over it. :P

You're sounding like me when I talk about C++. As far as Python being faster than C++...heh, maybe in certain situations, but anything you can do to optimize Python, can be done almost tenfold in C++. Bytecode or not, it's still interpreted. I believe Python might be able to emit C/C++ code, like Perl can, but there will always be an extra VM/interpreter layer, then the C runtime layer, etc. If Python could emit object code and use a linker, then yeah, you're talking a real performance gain.

Check the PyPy project sometime. Their goal is to do better than the current C/C++ compilers, by completely optimizing the code during something similar to just in time compilation. Even C/C++ compilers have to generalize some things that would work better if completely optimized. I'm not saying it can't be done with C/C++, I'm just saying that the C/C++ compilers currently don't do that at all.

And yes, after years of working with literally dozens upon dozens of languages, C and C++ included, I prefer Python. Less chance of horribly screwing something up, more control, easier and faster to build, and very readable. The PyPy project has shown that Python can match native C/C++ code, if not do better in many areas due to the limitations of the compilers that exist currently.

[d.vel.oper]

Check the PyPy project sometime. Their goal is to do better than the current C/C++ compilers, by completely optimizing the code during something similar to just in time compilation. Even C/C++ compilers have to generalize some things that would work better if completely optimized. I'm not saying it can't be done with C/C++, I'm just saying that the C/C++ compilers currently don't do that at all.

And yes, after years of working with literally dozens upon dozens of languages, C and C++ included, I prefer Python. Less chance of horribly screwing something up, more control, easier and faster to build, and very readable. The PyPy project has shown that Python can match native C/C++ code, if not do better in many areas due to the limitations of the compilers that exist currently.

I can has hardware acceleration plx? Maybe PyPy and a Boost.Python wrapper around OpenGL and D3D. I'm too high to work out implementation details, but uh, I am intrigued, because I DO like Python, especially with Boost.Python, but I'm currently working on stuff that's pretty close to the hardware. Yeah, gamedev, but I'm currently working on a long-going pet project that has to deal with perfect ring3 and ring0 Detour function patching, so I can write a general framework for modifying existing binaries in memory. Sorta has to deal with my 'hacking' toys, but more generalized. I was using Boost.Python to bind a scripting language to the C++/Win32 API backend, with some ASM mixins.

Mmmh, ranting now, but yeah, thanks for pointing me to PyPy.

[Xlorep DarkHelm]

I do think that Python works best, in a graphics-environment, when the underlying hardware interface is already built in something that can be a lot lower-level direct hardware control. For most things, Python works best as sort of the thing that ties everything else together.

PyPy is quite intriguing, they only recently got to a 1.0 release, however.

[d.vel.oper]

I do think that Python works best, in a graphics-environment, when the underlying hardware interface is already built in something that can be a lot lower-level direct hardware control. For most things, Python works best as sort of the thing that ties everything else together.

PyPy is quite intriguing, they only recently got to a 1.0 release, however.

Yeah, I noticed. My current 2D Direct3D 9-based engine implementation is easy enough to expose to Python (I don't care what you say, if you write CORRECT interface classes, it's pretty hard to break a C++ class). Especially via Boost.Python. So I'll just keep working with that until I run into a project where an architecture change would justify the time saved.

[Xlorep DarkHelm]

I don't care what you say, if you write CORRECT interface classes, it's pretty hard to break a C++ class.

true enough. But in a team environment, where you may have to dig through other peoples' code as well, you might be able to make your code relatively bug-free, but can you guarantee that the rest of your team will stick to that standard? Or what when "crunch time" comes into play and everyone's rushing to meet the deadline? What if the project was already completed, but you are put in the position of debugging, updating, and enhancing it? C++ is great. But honestly, I think there just are too many ways to do the same thing in it, and while that allows for an open toolbox and room for a lot of artistic style in the code, it is a double-edged sword, as it can make the code very.... arcane, difficult to work with what other people have done, and so forth.

Don't get me wrong, I like the ideals of C++, and in a single-developer environment where I have full control over the code personally, I like it. But I rarely am granted that luxury. Time tends to be a luxury I don't have, so rapid development tends to be far more of interest to me than what C++ offers.